-- Logs begin at Thu 2022-09-15 06:48:05 UTC, end at Thu 2022-09-15 07:57:11 UTC. -- Sep 15 07:18:58.293067 dv-cinder-824149-6-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 15 07:18:58.301739 dv-cinder-824149-6-iscsi devstack@keystone.service[81587]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 15 07:18:58.304036 dv-cinder-824149-6-iscsi devstack@keystone.service[81587]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Sep 15 07:18:58 2022] *** Sep 15 07:18:58.304036 dv-cinder-824149-6-iscsi devstack@keystone.service[81587]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 15 07:18:58.304036 dv-cinder-824149-6-iscsi devstack@keystone.service[81587]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 15 07:18:58.304036 dv-cinder-824149-6-iscsi devstack@keystone.service[81587]: nodename: dv-cinder-824149-6-iscsi Sep 15 07:18:58.304036 dv-cinder-824149-6-iscsi devstack@keystone.service[81587]: machine: x86_64 Sep 15 07:18:58.304036 dv-cinder-824149-6-iscsi devstack@keystone.service[81587]: clock source: unix Sep 15 07:18:58.304036 dv-cinder-824149-6-iscsi devstack@keystone.service[81587]: pcre jit disabled Sep 15 07:18:58.304036 dv-cinder-824149-6-iscsi devstack@keystone.service[81587]: detected number of CPU cores: 4 Sep 15 07:18:58.304036 dv-cinder-824149-6-iscsi devstack@keystone.service[81587]: current working directory: / Sep 15 07:18:58.304036 dv-cinder-824149-6-iscsi devstack@keystone.service[81587]: detected binary path: /usr/bin/uwsgi-core Sep 15 07:18:58.304036 dv-cinder-824149-6-iscsi devstack@keystone.service[81587]: your processes number limit is 47680 Sep 15 07:18:58.304619 dv-cinder-824149-6-iscsi devstack@keystone.service[81587]: your memory page size is 4096 bytes Sep 15 07:18:58.304619 dv-cinder-824149-6-iscsi devstack@keystone.service[81587]: detected max file descriptor number: 2048 Sep 15 07:18:58.304619 dv-cinder-824149-6-iscsi devstack@keystone.service[81587]: lock engine: pthread robust mutexes Sep 15 07:18:58.304619 dv-cinder-824149-6-iscsi devstack@keystone.service[81587]: thunder lock: enabled Sep 15 07:18:58.304619 dv-cinder-824149-6-iscsi devstack@keystone.service[81587]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 15 07:18:58.304619 dv-cinder-824149-6-iscsi devstack@keystone.service[81587]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 15 07:18:58.343892 dv-cinder-824149-6-iscsi devstack@keystone.service[81587]: Python main interpreter initialized at 0x55e0bd2839c0 Sep 15 07:18:58.343892 dv-cinder-824149-6-iscsi devstack@keystone.service[81587]: python threads support enabled Sep 15 07:18:58.343892 dv-cinder-824149-6-iscsi devstack@keystone.service[81587]: your server socket listen backlog is limited to 100 connections Sep 15 07:18:58.343892 dv-cinder-824149-6-iscsi devstack@keystone.service[81587]: your mercy for graceful operations on workers is 90 seconds Sep 15 07:18:58.344248 dv-cinder-824149-6-iscsi devstack@keystone.service[81587]: mapped 403077 bytes (393 KB) for 2 cores Sep 15 07:18:58.344322 dv-cinder-824149-6-iscsi devstack@keystone.service[81587]: *** Operational MODE: preforking *** Sep 15 07:18:58.344632 dv-cinder-824149-6-iscsi devstack@keystone.service[81587]: *** uWSGI is running in multiple interpreter mode *** Sep 15 07:18:58.344632 dv-cinder-824149-6-iscsi devstack@keystone.service[81587]: spawned uWSGI master process (pid: 81587) Sep 15 07:18:58.347780 dv-cinder-824149-6-iscsi devstack@keystone.service[81587]: spawned uWSGI worker 1 (pid: 81597, cores: 1) Sep 15 07:18:58.347780 dv-cinder-824149-6-iscsi devstack@keystone.service[81587]: spawned uWSGI worker 2 (pid: 81598, cores: 1) Sep 15 07:18:58.347780 dv-cinder-824149-6-iscsi devstack@keystone.service[81587]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 15 07:18:58.344835 dv-cinder-824149-6-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 15 07:19:02.720363 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 15 07:19:02.720363 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: from cryptography.utils import int_from_bytes Sep 15 07:19:02.721150 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 15 07:19:02.721150 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: from cryptography.utils import int_from_bytes Sep 15 07:19:02.722056 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 15 07:19:02.722056 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: from cryptography.utils import int_from_bytes Sep 15 07:19:02.722533 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 15 07:19:02.722533 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: from cryptography.utils import int_from_bytes Sep 15 07:19:02.777669 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 15 07:19:02.777840 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 15 07:19:02.777941 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 15 07:19:02.778036 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 15 07:19:02.778122 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] command line args: [] {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 15 07:19:02.778200 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] command line args: [] {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 15 07:19:02.778318 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 15 07:19:02.778404 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 15 07:19:02.778482 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ================================================================================ {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 15 07:19:02.778554 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ================================================================================ {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 15 07:19:02.778747 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] admin_token = **** {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.778835 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] admin_token = **** {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.779001 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] config_dir = [] {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.779082 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] config_dir = [] {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.779189 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.779268 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.779375 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] config_source = [] {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.779461 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] config_source = [] {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.779538 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] debug = True {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.779610 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] debug = True {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.779745 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.779998 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.780906 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.780989 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.781049 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.781119 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.781119 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.781234 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.781296 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.781358 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.781428 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] list_limit = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.781566 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] list_limit = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.781696 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] log_config_append = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.781774 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] log_config_append = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.781902 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.781983 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.782108 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] log_dir = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.782189 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] log_dir = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.782318 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] log_file = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.782394 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] log_file = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.782511 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.782591 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.782729 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.782837 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.782923 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.783001 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.783077 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.783150 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.783262 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.783352 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.783433 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.783510 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.783589 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.783694 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.783777 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.783857 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.783931 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.784040 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.784123 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.784199 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.784319 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.784403 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.784475 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.784558 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.784628 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.784738 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.784828 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.784900 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.785019 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.785148 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.785494 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.42/identity {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.786993 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.42/identity {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.787113 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] publish_errors = False {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.787233 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] publish_errors = False {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.787359 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.787474 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.787605 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.787718 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.787830 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.787952 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.788132 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.788270 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.788391 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.788505 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.788617 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] standard_threads = False {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.788739 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] standard_threads = False {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.788860 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.788974 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.789087 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.789207 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.789320 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.789441 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.789561 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] use_journal = False {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.789703 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] use_journal = False {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.789817 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] use_json = False {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.789947 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] use_json = False {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.790068 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] use_stderr = False {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.790192 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] use_stderr = False {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.790312 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] use_syslog = False {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.790426 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] use_syslog = False {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.790575 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.790688 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 07:19:02.790807 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.790930 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.791045 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.791165 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.791279 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.791398 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.791592 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.792247 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.792434 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.792572 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.792693 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.792917 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.793049 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.793170 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.793292 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.793412 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.793532 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.793681 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.793814 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.793930 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.794044 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.794157 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.794271 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.794385 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.794510 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.794625 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.794744 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.794859 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.794987 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.794987 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.795224 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.795341 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.795462 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.795583 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.795697 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.795852 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.796072 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.796196 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.796196 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.796412 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.796532 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.796647 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.796647 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.796871 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.796871 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.797247 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.797363 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.797485 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.797641 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.797760 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.797875 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.797992 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.798107 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.798220 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.798410 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.798529 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.798645 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.798765 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.798880 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.798994 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.799119 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.799234 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.799348 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.799468 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.799590 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.799713 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.799834 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.799948 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.800062 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.800182 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.800296 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.800421 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.800537 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.800652 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.800772 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.800887 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.801001 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.801131 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.801236 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.801336 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.801436 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.801530 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.804233 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.804353 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.804451 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.804545 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.804639 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.804739 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.804869 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.804978 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.805075 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.805171 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.805267 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.805362 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.805601 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.805700 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.805776 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.805897 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.805972 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.806088 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.806163 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.806241 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.806322 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.806394 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.806520 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.806597 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.806714 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.806789 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.806906 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.807021 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.807097 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.807218 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.807410 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.808013 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.808088 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.808154 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.808213 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.808278 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.808337 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.808395 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.808471 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.808538 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.808610 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.808691 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.808763 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.808839 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.808917 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.808984 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.809049 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.809121 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.809193 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.809259 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.809337 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.809418 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.809490 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.809558 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.809660 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.809783 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.809906 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.809990 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.810111 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.810187 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.810304 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.810385 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.810464 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.810548 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.810626 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.810704 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.810780 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.810857 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.810935 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.811082 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.811157 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.811157 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.811335 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.811410 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.811489 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.811561 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.811631 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.811701 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.811701 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.811834 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.811912 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.811993 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.812075 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.812153 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.812224 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.812294 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.812294 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.812426 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.812504 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.812577 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.812646 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.812716 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.812786 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.812855 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.812855 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.812995 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.813067 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.813136 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.813206 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.813275 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.813380 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.813502 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.813630 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.813714 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.813786 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.813856 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.813939 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.814013 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.814084 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.814154 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.814224 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.814297 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.814364 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.814534 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.814612 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.814684 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.814754 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.814824 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.814903 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.814977 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.815055 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.815128 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.815198 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.815269 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.815347 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.815420 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.815490 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.815561 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.815637 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] auth.external = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.815710 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.815787 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.815889 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.815974 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.816048 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.816118 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.816188 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.816259 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] auth.password = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.816328 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] auth.external = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.816407 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] auth.token = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.816480 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.816581 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.816717 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.816818 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.816961 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.817067 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.817199 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] auth.password = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.817299 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.817438 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] auth.token = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.817547 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.817710 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.817817 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.817954 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.818042 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.818208 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.818296 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] credential.caching = True {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.818446 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.818528 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.818709 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.818795 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.819043 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.819199 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.819343 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.819475 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.819605 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] credential.caching = True {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.819733 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.819860 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.819994 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.820130 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.820260 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.820384 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.820513 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.820642 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.820775 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.820912 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.821049 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.821185 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.821307 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.821436 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.821564 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.821733 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.821858 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.821987 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.822115 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.822244 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.822372 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.822576 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.822699 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] federation.caching = True {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.822699 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.822959 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.823163 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.823300 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.823481 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.823616 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.823781 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.823891 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.823976 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.823976 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.824116 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] federation.caching = True {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.824189 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.824260 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.824336 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.824409 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.824480 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.824558 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.824631 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.824702 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.824773 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.824844 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.824916 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] identity.caching = True {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.825084 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.825166 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.825329 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.825409 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.825569 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.825706 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.833100 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.833200 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.833279 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.833366 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.833442 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.833513 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.833607 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.833682 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] identity.caching = True {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.833753 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.833824 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.833894 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.833963 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.834049 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.834129 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.834201 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.834272 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.834342 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.834423 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.834495 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.834566 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.834636 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.834706 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.834775 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.834853 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.834925 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.834996 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.835066 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.835136 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.835207 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.835285 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.835357 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.835428 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.835498 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.835567 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.835637 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.835714 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.835791 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.835862 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.835933 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.836002 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.836072 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.836149 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.836254 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.836328 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.836388 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.836457 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.836532 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.836612 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.836684 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.836754 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.836824 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.836893 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.836962 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.837040 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.837111 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.837180 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.837250 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.837320 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.837390 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.837511 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.837637 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.837716 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.837797 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.837869 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.837940 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.838023 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.838097 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.838168 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.838238 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.838308 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.838387 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.838466 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.838544 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.838616 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.838712 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.838778 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.838837 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.838904 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.838964 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.839295 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.839360 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.839419 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.839477 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.839544 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.839604 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.839669 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.839728 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.839786 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.839843 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.839909 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.839969 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.840027 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.840084 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.840141 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.840198 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.user = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.840262 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.840322 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.840380 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.840443 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.840503 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.840560 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.840626 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.840686 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.840744 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.840801 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.840858 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.840928 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.840994 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.841054 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.841112 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.user = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.841169 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.841169 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.841280 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.841345 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.841404 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.841462 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.841519 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.841616 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.841781 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.841851 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.841910 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.841967 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.841967 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.842069 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.842069 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.842181 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.842247 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.842305 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.842305 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.842410 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.842410 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.842523 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.842523 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.842625 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.842625 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.842727 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.842727 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.842838 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.842914 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.842972 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.843029 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.843086 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.843150 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.843210 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.843267 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.843324 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.843380 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.843438 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.843503 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.843562 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.843625 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.843684 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.843852 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.843930 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.844069 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.844145 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.844300 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.844377 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.844518 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.844595 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.844772 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.844850 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.845011 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.845088 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.845252 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.845328 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.845462 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] resource.caching = True {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.845572 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.845739 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.845841 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.845987 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.846059 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.850210 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.850332 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.850332 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.850528 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.850603 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.850675 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.850675 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.850822 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.850822 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.850951 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.850951 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.851079 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.851158 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.851274 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] resource.caching = True {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.851356 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] role.caching = True {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.851427 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.851498 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] role.driver = sql {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.851568 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.851647 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.851721 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.851791 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.851861 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.851932 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.852003 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.852081 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.852154 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.852225 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.852295 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.852366 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.852436 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.852515 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.852588 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.852659 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.852729 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] role.caching = True {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.852799 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.852869 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] role.driver = sql {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.852947 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.853021 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.853021 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.853147 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.853218 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.853288 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.853288 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.853456 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.853518 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.853614 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.853614 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.853814 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.853891 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.853963 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.854034 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.854104 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.854677 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.854769 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.854872 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.854985 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.855061 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.855133 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.855204 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.855286 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.855372 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.855476 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.855604 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.855688 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.855688 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.855828 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.855901 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.855972 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.856137 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.856220 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.856293 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.856293 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.856488 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.856566 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.856637 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.856708 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.856779 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.856858 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.856858 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.856989 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.857060 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.857131 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.857202 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] token.caching = True {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.857280 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.857353 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.857423 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.857494 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.857564 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.857663 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.857749 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.857823 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.857894 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.857965 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.858035 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.858114 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.858261 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.858346 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.858514 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.858594 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.858825 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.858997 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.859080 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.859152 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.859223 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.859349 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.859431 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] token.caching = True {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.859511 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.859675 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.859756 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.859932 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.860012 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.860174 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.860268 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.860435 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.860527 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.860687 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.860767 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.860928 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.861008 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] database.connection = **** {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.861177 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.861265 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.861422 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.861501 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.861685 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.861770 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.861931 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.862025 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.862167 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.862248 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.862406 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.862485 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.862643 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.862730 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.862881 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.862962 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.863127 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.863206 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.863364 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.863454 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.863610 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.863691 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.863845 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.863924 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] database.connection = **** {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.864093 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.864173 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.864359 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.864440 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.864590 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.864670 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.864837 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.864916 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.865076 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.865158 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.865312 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.865390 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.865485 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 15 07:19:02.865565 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.865807 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.866058 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.866299 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.866549 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.866789 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.867040 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.867282 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.867532 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.867767 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.868013 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.868253 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 07:19:02.868426 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 15 07:19:02.895524 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 07:19:02.895754 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 07:19:02.895978 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 07:19:02.904531 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 07:19:02.904765 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 07:19:02.904997 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 07:19:02.905084 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 07:19:02.913384 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 07:19:02.935885 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 07:19:02.936114 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 07:19:02.940153 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 07:19:02.940392 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 07:19:02.997040 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 07:19:02.997769 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 07:19:03.046376 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 07:19:03.046600 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 07:19:03.051183 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 07:19:03.051425 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 07:19:03.220257 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 07:19:03.220448 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 07:19:03.220618 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 07:19:03.220703 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 07:19:03.220779 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 07:19:03.220890 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 07:19:03.220975 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 07:19:03.221107 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 07:19:03.221190 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 07:19:03.221329 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 07:19:03.221398 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 07:19:03.221550 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 07:19:03.245919 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 07:19:03.246152 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 07:19:03.246309 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 07:19:03.246397 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 07:19:03.246551 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 07:19:03.246684 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 07:19:03.251806 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 07:19:03.251903 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 07:19:03.251979 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 07:19:03.252052 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 07:19:03.252113 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 07:19:03.252251 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 07:19:03.252329 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 07:19:03.252329 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 07:19:03.262580 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 07:19:03.263082 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 07:19:03.275267 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 07:19:03.275444 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 07:19:03.275524 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 07:19:03.275597 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 07:19:03.275708 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 07:19:03.275790 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 07:19:03.275930 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 07:19:03.276011 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 07:19:03.276677 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 07:19:03.276812 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 07:19:03.276991 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 07:19:03.277085 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 07:19:03.277209 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 07:19:03.277402 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 07:19:03.294396 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.294605 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 15 07:19:03.294909 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.294993 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.295123 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 15 07:19:03.295204 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 15 07:19:03.295399 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.295478 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.295663 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.295745 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 15 07:19:03.295922 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.296003 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.296179 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.296266 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.296441 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.296704 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.306070 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.306351 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.306621 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.306716 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.306905 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.306985 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.307182 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.307501 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.314114 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 07:19:03.314872 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81597) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 07:19:03.317045 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 07:19:03.317326 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.317641 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.317803 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81597) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 07:19:03.317927 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.318096 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.318211 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.318386 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.318466 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.318675 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.318757 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.318963 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.319043 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.319228 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.319315 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.319501 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.319581 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.319768 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.319846 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.320042 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.320319 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.320591 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.343077 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.344360 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.344546 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 07:19:03.344832 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.345882 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81597) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 07:19:03.346174 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.349544 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 07:19:03.349852 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.350135 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.350785 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81597) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 07:19:03.351075 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.351476 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.357043 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 07:19:03.357328 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.358795 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81597) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 07:19:03.359090 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.362963 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 07:19:03.363056 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81597) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 07:19:03.363257 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.363359 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.363542 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.363624 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.363827 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.364103 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.364380 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.364650 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.364919 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.365591 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.365904 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.366199 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.366491 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.366779 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.380333 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.380609 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.382456 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.382743 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.384403 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 07:19:03.387375 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81597) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 07:19:03.388248 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.388547 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.390475 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.390892 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.392674 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 07:19:03.395081 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81597) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 07:19:03.395772 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 07:19:03.398169 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81597) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 07:19:03.399011 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.399310 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.399595 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.399878 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.400157 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.400445 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.401302 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.401880 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.402179 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.402584 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.402873 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.403157 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.416172 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 07:19:03.416460 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.417253 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.417513 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.419492 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81597) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 07:19:03.419785 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.420062 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.420332 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.424305 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.425853 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.426987 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.427618 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.427898 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.428886 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.430425 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.430559 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 07:19:03.430693 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.430837 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.431092 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.431376 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.431765 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.433895 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81597) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 07:19:03.434194 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.434483 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.434770 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.435073 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.442981 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 07:19:03.443267 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.443549 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.443830 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.444209 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.444506 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.444788 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.446307 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81597) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 07:19:03.446602 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.446888 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.447172 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.447458 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.447737 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.448614 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.460537 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 07:19:03.463436 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 07:19:03.464662 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81597) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 07:19:03.466569 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.467902 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81597) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 07:19:03.468057 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.469520 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 07:19:03.469831 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.470233 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.470808 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.472256 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.473755 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81597) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 07:19:03.474047 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.474336 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.476526 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.478008 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 07:19:03.481400 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.481822 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.481935 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.482112 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.482388 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.483399 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81597) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 07:19:03.486320 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.486617 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.486906 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.487186 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.490124 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 07:19:03.490422 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.490715 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.495318 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81597) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 07:19:03.495685 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.495976 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.497751 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 07:19:03.498038 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.498320 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.498604 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.498875 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.499145 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.499417 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.499702 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.499972 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.500238 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.500668 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.500947 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.501213 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.501472 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.502906 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81597) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 07:19:03.503151 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.503437 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.503720 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.504002 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.504279 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.504554 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.504838 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.505118 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.505400 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.505725 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.505994 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.506406 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.506695 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 07:19:03.535669 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55e0bd2839c0 pid: 81598 (default app) Sep 15 07:19:03.538925 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c81b899-0947-442a-b362-a7121b48cc0d None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:03.539216 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c81b899-0947-442a-b362-a7121b48cc0d None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:03.539491 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c81b899-0947-442a-b362-a7121b48cc0d None None] PATH_INFO: `/v3` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:03.541424 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55e0bd2839c0 pid: 81597 (default app) Sep 15 07:19:03.544439 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1/1] 10.222.0.42 () {50 vars in 791 bytes} [Thu Sep 15 07:19:03 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:19:11.345493 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c13e6df-30e6-4261-92e8-916cd42257b2 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:11.346023 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c13e6df-30e6-4261-92e8-916cd42257b2 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:11.346110 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c13e6df-30e6-4261-92e8-916cd42257b2 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:11.350319 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1/2] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:19:11 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 07:19:11.356294 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbbcedf-9e28-48fe-adee-ec5d3f8a6424 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:11.356740 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbbcedf-9e28-48fe-adee-ec5d3f8a6424 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:11.356839 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbbcedf-9e28-48fe-adee-ec5d3f8a6424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:11.432324 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: INFO dbcounter [None req-ebbbcedf-9e28-48fe-adee-ec5d3f8a6424 None None] Registered counter for database keystone Sep 15 07:19:11.451179 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG oslo_db.sqlalchemy.engines [None req-ebbbcedf-9e28-48fe-adee-ec5d3f8a6424 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81598) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 15 07:19:11.457340 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writer thread running {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 15 07:19:11.635629 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG passlib.handlers.bcrypt [None req-ebbbcedf-9e28-48fe-adee-ec5d3f8a6424 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81598) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 15 07:19:11.635985 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG passlib.handlers.bcrypt [None req-ebbbcedf-9e28-48fe-adee-ec5d3f8a6424 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81598) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 15 07:19:11.693910 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-ebbbcedf-9e28-48fe-adee-ec5d3f8a6424 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:19:11.695493 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-ebbbcedf-9e28-48fe-adee-ec5d3f8a6424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:19:11.820899 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 2/3] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:19:11 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 466 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 15 07:19:12.309001 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c13e6df-30e6-4261-92e8-916cd42257b2 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:12.380139 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: INFO dbcounter [None req-0c13e6df-30e6-4261-92e8-916cd42257b2 None None] Registered counter for database keystone Sep 15 07:19:12.401286 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG oslo_db.sqlalchemy.engines [None req-0c13e6df-30e6-4261-92e8-916cd42257b2 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81597) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 15 07:19:12.407503 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writer thread running {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 15 07:19:12.527991 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c13e6df-30e6-4261-92e8-916cd42257b2 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:12.532239 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a809188-c288-4062-afef-c0e07ec0aa7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:12.532875 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a809188-c288-4062-afef-c0e07ec0aa7b admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:12.533163 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a809188-c288-4062-afef-c0e07ec0aa7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:12.533442 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a809188-c288-4062-afef-c0e07ec0aa7b admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:12.541598 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a809188-c288-4062-afef-c0e07ec0aa7b admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:12.593769 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: WARNING keystone.common.rbac_enforcer.enforcer [None req-8a809188-c288-4062-afef-c0e07ec0aa7b admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 15 07:19:12.594073 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a809188-c288-4062-afef-c0e07ec0aa7b admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:12.603406 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 2/4] 10.222.0.42 () {58 vars in 1324 bytes} [Thu Sep 15 07:19:12 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 15 07:19:12.609892 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbbcedf-9e28-48fe-adee-ec5d3f8a6424 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:12.627342 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbbcedf-9e28-48fe-adee-ec5d3f8a6424 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:12.631665 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb3f1c8-fa39-4c7a-8432-54fcd2a6b0e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:12.632782 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb3f1c8-fa39-4c7a-8432-54fcd2a6b0e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:12.633062 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb3f1c8-fa39-4c7a-8432-54fcd2a6b0e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:12.633330 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb3f1c8-fa39-4c7a-8432-54fcd2a6b0e7 admin admin] PATH_INFO: `/v3/projects/918580af0a81452f8da95d60256aaea0` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:12.634941 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb3f1c8-fa39-4c7a-8432-54fcd2a6b0e7 admin admin] RBAC: Authorizing `identity:get_project(project_id=918580af0a81452f8da95d60256aaea0)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:12.688119 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: WARNING keystone.common.rbac_enforcer.enforcer [None req-8bb3f1c8-fa39-4c7a-8432-54fcd2a6b0e7 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 15 07:19:12.688461 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb3f1c8-fa39-4c7a-8432-54fcd2a6b0e7 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:12.692941 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 3/5] 10.222.0.42 () {56 vars in 1207 bytes} [Thu Sep 15 07:19:12 2022] GET /identity/v3/projects/918580af0a81452f8da95d60256aaea0 => generated 344 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:19:14.803551 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da94f45-08ff-4903-9085-359b2836e40e None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:14.804103 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da94f45-08ff-4903-9085-359b2836e40e None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:14.804103 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da94f45-08ff-4903-9085-359b2836e40e None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:14.808792 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 3/6] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:19:14 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:19:14.814484 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee38ef6-93e9-4695-8c19-5e5e9dd7f6e0 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:14.814889 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee38ef6-93e9-4695-8c19-5e5e9dd7f6e0 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:14.815051 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee38ef6-93e9-4695-8c19-5e5e9dd7f6e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:14.834861 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-1ee38ef6-93e9-4695-8c19-5e5e9dd7f6e0 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:19:14.836402 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-1ee38ef6-93e9-4695-8c19-5e5e9dd7f6e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:19:14.844790 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 4/7] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:19:14 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 15 07:19:15.110929 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da94f45-08ff-4903-9085-359b2836e40e None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:15.123903 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da94f45-08ff-4903-9085-359b2836e40e None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:15.134472 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ef9a68-942b-4ea7-9c20-1df58356ea59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:15.135091 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ef9a68-942b-4ea7-9c20-1df58356ea59 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:15.135383 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ef9a68-942b-4ea7-9c20-1df58356ea59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:15.135669 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ef9a68-942b-4ea7-9c20-1df58356ea59 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:15.144373 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ef9a68-942b-4ea7-9c20-1df58356ea59 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:15.146197 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ef9a68-942b-4ea7-9c20-1df58356ea59 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:15.155125 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 4/8] 10.222.0.42 () {58 vars in 1324 bytes} [Thu Sep 15 07:19:15 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 07:19:15.160685 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee38ef6-93e9-4695-8c19-5e5e9dd7f6e0 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:15.169788 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee38ef6-93e9-4695-8c19-5e5e9dd7f6e0 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:15.174129 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97d431e-9292-4513-994b-dee0c5eb2331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:15.174755 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97d431e-9292-4513-994b-dee0c5eb2331 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:15.175039 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97d431e-9292-4513-994b-dee0c5eb2331 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:15.175315 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97d431e-9292-4513-994b-dee0c5eb2331 admin admin] PATH_INFO: `/v3/users/06cea60dcdfb4c5ead6de8db2f0fd82f` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:15.176633 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97d431e-9292-4513-994b-dee0c5eb2331 admin admin] RBAC: Authorizing `identity:get_user(user_id=06cea60dcdfb4c5ead6de8db2f0fd82f)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:15.178774 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97d431e-9292-4513-994b-dee0c5eb2331 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:15.180299 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 5/9] 10.222.0.42 () {56 vars in 1198 bytes} [Thu Sep 15 07:19:15 2022] GET /identity/v3/users/06cea60dcdfb4c5ead6de8db2f0fd82f => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:19:17.398630 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce8e7e8-a1ee-41f2-bae9-79dd0cc236e9 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:17.399204 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce8e7e8-a1ee-41f2-bae9-79dd0cc236e9 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:17.399204 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce8e7e8-a1ee-41f2-bae9-79dd0cc236e9 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:17.400519 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 5/10] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:19:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:19:17.406592 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a4ce7c-0407-4d83-9768-1579110ed66f None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:17.407018 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a4ce7c-0407-4d83-9768-1579110ed66f None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:17.407288 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a4ce7c-0407-4d83-9768-1579110ed66f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:17.425861 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-17a4ce7c-0407-4d83-9768-1579110ed66f None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:19:17.427027 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-17a4ce7c-0407-4d83-9768-1579110ed66f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:19:17.428534 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59d4fc7-6fe3-4f0c-b0be-e8cf03bdd978 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:17.428822 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59d4fc7-6fe3-4f0c-b0be-e8cf03bdd978 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:17.429105 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59d4fc7-6fe3-4f0c-b0be-e8cf03bdd978 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:17.430482 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 6/11] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:19:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 07:19:17.435525 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7d823a-1cf4-46d4-9dfc-ecdb533f30a9 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:17.435816 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7d823a-1cf4-46d4-9dfc-ecdb533f30a9 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:17.436096 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7d823a-1cf4-46d4-9dfc-ecdb533f30a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:17.478248 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG passlib.handlers.bcrypt [None req-fa7d823a-1cf4-46d4-9dfc-ecdb533f30a9 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81597) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 15 07:19:17.478635 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG passlib.handlers.bcrypt [None req-fa7d823a-1cf4-46d4-9dfc-ecdb533f30a9 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81597) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 15 07:19:17.509445 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 6/12] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:19:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 07:19:17.530224 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-fa7d823a-1cf4-46d4-9dfc-ecdb533f30a9 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:19:17.531857 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-fa7d823a-1cf4-46d4-9dfc-ecdb533f30a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:19:17.639224 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 7/13] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:19:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 07:19:17.776936 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a4ce7c-0407-4d83-9768-1579110ed66f None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:17.801406 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:17.883020 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a4ce7c-0407-4d83-9768-1579110ed66f None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:17.916315 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5b63aa-01fc-4386-9f10-94ae7fb1a924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:17.916949 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5b63aa-01fc-4386-9f10-94ae7fb1a924 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:17.917257 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5b63aa-01fc-4386-9f10-94ae7fb1a924 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:17.917534 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5b63aa-01fc-4386-9f10-94ae7fb1a924 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:17.925268 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7d823a-1cf4-46d4-9dfc-ecdb533f30a9 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:17.929441 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5b63aa-01fc-4386-9f10-94ae7fb1a924 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:19:17.938021 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-0a5b63aa-01fc-4386-9f10-94ae7fb1a924 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 07:19:17.938021 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:19:17.938021 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:19:17.938021 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:19:17.938021 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:19:17.938021 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:19:17.938021 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:19:17.938021 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:19:17.938021 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:19:17.938021 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:19:17.938021 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:19:17.938021 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:19:17.938021 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:19:17.938021 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:19:17.938021 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:19:17.938021 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:19:17.938967 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:19:17.938967 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:19:17.938967 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:19:17.938967 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:19:17.938967 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:19:17.938967 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:19:17.938967 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:19:17.938967 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:19:17.938967 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:19:17.938967 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:19:17.938967 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:19:17.938967 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:19:17.938967 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:19:17.938967 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:19:17.938967 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:19:17.938967 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:19:17.939786 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:19:17.939786 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:19:17.939786 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:19:17.939786 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:19:17.939786 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:19:17.939786 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:19:17.939786 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:19:17.939786 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:19:17.939786 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:19:17.939786 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:19:17.939786 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 07:19:17.939786 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:19:17.939786 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 7/14] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 15 07:19:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 07:19:17.946247 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5b63aa-01fc-4386-9f10-94ae7fb1a924 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:18.029032 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7d823a-1cf4-46d4-9dfc-ecdb533f30a9 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:18.029490 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:18.031393 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:18.047718 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5b63aa-01fc-4386-9f10-94ae7fb1a924 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:18.065677 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c473fd-b108-49a6-b759-2b641922bd9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:18.066545 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c473fd-b108-49a6-b759-2b641922bd9c None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:18.066833 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c473fd-b108-49a6-b759-2b641922bd9c None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:18.067109 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c473fd-b108-49a6-b759-2b641922bd9c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:18.082711 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa29bcf-5663-4b33-a58a-82d546f86a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:18.083321 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa29bcf-5663-4b33-a58a-82d546f86a76 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:18.083606 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa29bcf-5663-4b33-a58a-82d546f86a76 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:18.083881 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa29bcf-5663-4b33-a58a-82d546f86a76 None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:18.084537 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa29bcf-5663-4b33-a58a-82d546f86a76 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:19:18.084840 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa29bcf-5663-4b33-a58a-82d546f86a76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:18.086273 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa29bcf-5663-4b33-a58a-82d546f86a76 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:18.098442 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 8/15] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 15 07:19:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:19:18.104019 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa29bcf-5663-4b33-a58a-82d546f86a76 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:18.135130 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c473fd-b108-49a6-b759-2b641922bd9c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:18.136812 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c473fd-b108-49a6-b759-2b641922bd9c None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:18.201358 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa29bcf-5663-4b33-a58a-82d546f86a76 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:18.237893 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d51d9a1-3f32-40bb-90bc-a5470b28fa30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:18.238525 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d51d9a1-3f32-40bb-90bc-a5470b28fa30 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:18.238810 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d51d9a1-3f32-40bb-90bc-a5470b28fa30 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:18.238948 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 8/16] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 15 07:19:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 15 07:19:18.239157 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d51d9a1-3f32-40bb-90bc-a5470b28fa30 None admin] PATH_INFO: `/v3/users/06cea60dcdfb4c5ead6de8db2f0fd82f` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:18.240370 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d51d9a1-3f32-40bb-90bc-a5470b28fa30 None admin] RBAC: Authorizing `identity:get_user(user_id=06cea60dcdfb4c5ead6de8db2f0fd82f)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:18.241862 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d51d9a1-3f32-40bb-90bc-a5470b28fa30 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:18.243549 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 9/17] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 15 07:19:18 2022] GET /identity/v3/users/06cea60dcdfb4c5ead6de8db2f0fd82f => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:19:18.244357 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c473fd-b108-49a6-b759-2b641922bd9c None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:18.247899 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:18.249450 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d51d9a1-3f32-40bb-90bc-a5470b28fa30 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:18.265808 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:18.345540 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c473fd-b108-49a6-b759-2b641922bd9c None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:18.350909 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d51d9a1-3f32-40bb-90bc-a5470b28fa30 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:18.380190 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105763ba-b03f-4990-90d9-19a5a68c236b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:18.381051 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105763ba-b03f-4990-90d9-19a5a68c236b None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:18.381339 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105763ba-b03f-4990-90d9-19a5a68c236b None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:18.381650 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105763ba-b03f-4990-90d9-19a5a68c236b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:18.385035 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5288f2d4-00f1-46e4-a6ea-6c97139602ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:18.385689 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5288f2d4-00f1-46e4-a6ea-6c97139602ab None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:18.386545 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5288f2d4-00f1-46e4-a6ea-6c97139602ab None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:18.386966 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5288f2d4-00f1-46e4-a6ea-6c97139602ab None admin] PATH_INFO: `/v3/domains/default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:18.388077 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5288f2d4-00f1-46e4-a6ea-6c97139602ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:18.389559 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5288f2d4-00f1-46e4-a6ea-6c97139602ab None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:18.391126 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 10/18] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:19:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:19:18.396760 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5288f2d4-00f1-46e4-a6ea-6c97139602ab None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:18.400879 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105763ba-b03f-4990-90d9-19a5a68c236b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:18.402569 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105763ba-b03f-4990-90d9-19a5a68c236b None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:18.421810 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-105763ba-b03f-4990-90d9-19a5a68c236b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:19:18.421810 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:19:18.421810 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:19:18.421810 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:19:18.421810 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:19:18.421810 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:19:18.421810 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:19:18.421810 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:19:18.421810 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:19:18.421810 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:19:18.421810 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:19:18.421810 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:19:18.421810 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 07:19:18.421810 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 07:19:18.421810 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 07:19:18.421810 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 07:19:18.422707 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:19:18.422707 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:19:18.422707 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:19:18.422707 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:19:18.422707 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:19:18.422707 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:19:18.422707 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:19:18.422707 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:19:18.422707 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:19:18.422707 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:19:18.422707 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:19:18.422707 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:19:18.422707 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:19:18.422707 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:19:18.422707 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:19:18.422707 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:19:18.422707 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 07:19:18.423492 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 07:19:18.423492 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:19:18.423492 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:19:18.423492 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 9/19] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:19:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 07:19:18.428840 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105763ba-b03f-4990-90d9-19a5a68c236b None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:18.478044 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:18.485076 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:18.501309 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5288f2d4-00f1-46e4-a6ea-6c97139602ab None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:18.531956 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105763ba-b03f-4990-90d9-19a5a68c236b None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:18.533509 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8b20ec-bb69-4833-8622-c13e49ee7026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:18.534152 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8b20ec-bb69-4833-8622-c13e49ee7026 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:18.534433 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8b20ec-bb69-4833-8622-c13e49ee7026 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:18.534707 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8b20ec-bb69-4833-8622-c13e49ee7026 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:18.535426 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8b20ec-bb69-4833-8622-c13e49ee7026 None admin] RBAC: Adding query filter params (role.id=6bbb643e0d8d4bc0a9b27c26edf59216, scope.domain.id=default, user.id=06cea60dcdfb4c5ead6de8db2f0fd82f) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:19:18.535737 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8b20ec-bb69-4833-8622-c13e49ee7026 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:18.537179 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8b20ec-bb69-4833-8622-c13e49ee7026 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:18.544615 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 11/20] 10.222.0.42 () {56 vars in 1322 bytes} [Thu Sep 15 07:19:18 2022] GET /identity/v3/role_assignments?user.id=06cea60dcdfb4c5ead6de8db2f0fd82f&scope.domain.id=default&role.id=6bbb643e0d8d4bc0a9b27c26edf59216 => generated 233 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:19:18.568873 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4accdf-507f-448a-8f2f-f8acaa10b8ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:18.569502 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4accdf-507f-448a-8f2f-f8acaa10b8ac None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:18.569818 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4accdf-507f-448a-8f2f-f8acaa10b8ac None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:18.570099 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4accdf-507f-448a-8f2f-f8acaa10b8ac None admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:18.570773 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4accdf-507f-448a-8f2f-f8acaa10b8ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:19:18.571080 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4accdf-507f-448a-8f2f-f8acaa10b8ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:18.572749 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4accdf-507f-448a-8f2f-f8acaa10b8ac None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:18.593378 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 10/21] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 15 07:19:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:19:20.931585 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e28fdd-6b90-4a15-9a60-427b25999ade None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:20.932112 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e28fdd-6b90-4a15-9a60-427b25999ade None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:20.932112 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e28fdd-6b90-4a15-9a60-427b25999ade None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:20.933764 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 12/22] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:19:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:19:20.940559 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1584d0c-2770-4da1-86c0-9ad2d64ca01a None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:20.940559 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1584d0c-2770-4da1-86c0-9ad2d64ca01a None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:20.940559 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1584d0c-2770-4da1-86c0-9ad2d64ca01a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:20.964216 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-f1584d0c-2770-4da1-86c0-9ad2d64ca01a None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:19:20.965538 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-f1584d0c-2770-4da1-86c0-9ad2d64ca01a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:19:21.060634 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 11/23] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:19:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 07:19:21.388449 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bfe0bb-b332-4e16-9013-fa93ae45f5c1 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:21.389157 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bfe0bb-b332-4e16-9013-fa93ae45f5c1 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:21.389673 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bfe0bb-b332-4e16-9013-fa93ae45f5c1 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:21.391221 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 13/24] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:19:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:19:21.396605 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b045808-33be-438d-a62d-7c3f52e782e0 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:21.396900 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b045808-33be-438d-a62d-7c3f52e782e0 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:21.397183 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b045808-33be-438d-a62d-7c3f52e782e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:21.426129 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-5b045808-33be-438d-a62d-7c3f52e782e0 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:19:21.427345 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-5b045808-33be-438d-a62d-7c3f52e782e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:19:21.491852 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:21.523425 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 12/25] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:19:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 07:19:21.605195 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bfe0bb-b332-4e16-9013-fa93ae45f5c1 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:21.734953 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bfe0bb-b332-4e16-9013-fa93ae45f5c1 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:21.789091 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ae857d-262a-4dae-b4b0-5274b2010077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:21.790224 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ae857d-262a-4dae-b4b0-5274b2010077 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:21.790718 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ae857d-262a-4dae-b4b0-5274b2010077 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:21.791202 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ae857d-262a-4dae-b4b0-5274b2010077 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:21.806985 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ae857d-262a-4dae-b4b0-5274b2010077 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:19:21.813254 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-19ae857d-262a-4dae-b4b0-5274b2010077 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 07:19:21.813254 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:19:21.813254 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:19:21.813254 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:19:21.813254 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:19:21.813254 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:19:21.813254 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:19:21.813254 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:19:21.813254 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:19:21.813254 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:19:21.813254 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:19:21.813254 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:19:21.813254 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:19:21.813254 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:19:21.813254 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:19:21.813254 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:19:21.814139 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:19:21.814139 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:19:21.814139 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:19:21.814139 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:19:21.814139 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:19:21.814139 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:19:21.814139 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:19:21.814139 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:19:21.814139 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:19:21.814139 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:19:21.814139 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:19:21.814139 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:19:21.814139 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:19:21.814139 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:19:21.814139 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:19:21.814139 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:19:21.814896 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:19:21.814896 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:19:21.814896 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:19:21.814896 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:19:21.814896 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:19:21.814896 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:19:21.814896 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:19:21.814896 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:19:21.814896 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:19:21.814896 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:19:21.814896 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 07:19:21.814896 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:19:21.817137 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 14/26] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 15 07:19:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:19:21.826268 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b045808-33be-438d-a62d-7c3f52e782e0 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:21.826676 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9327d5f-f5f1-429d-b4ab-9180aec37adc None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:21.827083 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9327d5f-f5f1-429d-b4ab-9180aec37adc None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:21.827362 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9327d5f-f5f1-429d-b4ab-9180aec37adc None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:21.830131 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 15/27] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:19:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:19:21.837917 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5983e107-925f-483d-bde4-50c55e1e58fb None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:21.838211 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5983e107-925f-483d-bde4-50c55e1e58fb None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:21.838464 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5983e107-925f-483d-bde4-50c55e1e58fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:21.852202 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:21.864893 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-5983e107-925f-483d-bde4-50c55e1e58fb None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:19:21.866348 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-5983e107-925f-483d-bde4-50c55e1e58fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:19:21.987738 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 16/28] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:19:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 07:19:21.991499 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5983e107-925f-483d-bde4-50c55e1e58fb None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:22.049516 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b045808-33be-438d-a62d-7c3f52e782e0 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:22.115706 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2721376e-9765-4e7c-9560-ec47f1825fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:22.116859 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2721376e-9765-4e7c-9560-ec47f1825fdd None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:22.121972 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2721376e-9765-4e7c-9560-ec47f1825fdd None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:22.122265 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2721376e-9765-4e7c-9560-ec47f1825fdd None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:22.122974 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2721376e-9765-4e7c-9560-ec47f1825fdd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:19:22.123280 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2721376e-9765-4e7c-9560-ec47f1825fdd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:22.124676 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2721376e-9765-4e7c-9560-ec47f1825fdd None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:22.149603 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 13/29] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 15 07:19:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:19:22.150868 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ac8b6b-28b5-42d7-b067-0708e8ed65dd None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:22.151161 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ac8b6b-28b5-42d7-b067-0708e8ed65dd None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:22.151440 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ac8b6b-28b5-42d7-b067-0708e8ed65dd None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:22.153429 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 14/30] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:19:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:19:22.158474 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ac8b6b-28b5-42d7-b067-0708e8ed65dd None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:22.160933 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5983e107-925f-483d-bde4-50c55e1e58fb None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:22.242370 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c49b159-b4b7-4bf3-8b26-a85428ecd170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:22.242948 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c49b159-b4b7-4bf3-8b26-a85428ecd170 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:22.243232 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c49b159-b4b7-4bf3-8b26-a85428ecd170 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:22.243512 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c49b159-b4b7-4bf3-8b26-a85428ecd170 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:22.248838 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:22.251917 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:22.274640 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c49b159-b4b7-4bf3-8b26-a85428ecd170 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:22.276155 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c49b159-b4b7-4bf3-8b26-a85428ecd170 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:22.302830 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-6c49b159-b4b7-4bf3-8b26-a85428ecd170 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:19:22.302830 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:19:22.302830 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:19:22.302830 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:19:22.302830 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:19:22.302830 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:19:22.302830 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:19:22.302830 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:19:22.302830 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:19:22.302830 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:19:22.302830 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:19:22.302830 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:19:22.302830 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 07:19:22.302830 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 07:19:22.302830 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 07:19:22.302830 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 07:19:22.304016 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:19:22.304016 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:19:22.304016 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:19:22.304016 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:19:22.304016 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:19:22.304016 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:19:22.304016 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:19:22.304016 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:19:22.304016 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:19:22.304016 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:19:22.304016 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:19:22.304016 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:19:22.304016 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:19:22.304016 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:19:22.304016 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:19:22.304016 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:19:22.304016 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 07:19:22.304885 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 07:19:22.304885 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:19:22.304885 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:19:22.305051 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ac8b6b-28b5-42d7-b067-0708e8ed65dd None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:22.305118 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 17/31] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:19:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:19:22.306020 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d09bcaf-3422-4d74-bf70-c9aa4cd452c5 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:22.306020 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d09bcaf-3422-4d74-bf70-c9aa4cd452c5 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:22.306020 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d09bcaf-3422-4d74-bf70-c9aa4cd452c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:22.337313 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-9d09bcaf-3422-4d74-bf70-c9aa4cd452c5 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:19:22.338827 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-9d09bcaf-3422-4d74-bf70-c9aa4cd452c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:19:22.343650 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a6a674-1a4a-48d6-9d1a-1418953b28d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:22.344303 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a6a674-1a4a-48d6-9d1a-1418953b28d3 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:22.344586 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a6a674-1a4a-48d6-9d1a-1418953b28d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:22.344866 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a6a674-1a4a-48d6-9d1a-1418953b28d3 None admin] PATH_INFO: `/v3/users/06cea60dcdfb4c5ead6de8db2f0fd82f` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:22.346543 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a6a674-1a4a-48d6-9d1a-1418953b28d3 None admin] RBAC: Authorizing `identity:get_user(user_id=06cea60dcdfb4c5ead6de8db2f0fd82f)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:22.348810 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a6a674-1a4a-48d6-9d1a-1418953b28d3 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:22.351415 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 15/32] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 15 07:19:22 2022] GET /identity/v3/users/06cea60dcdfb4c5ead6de8db2f0fd82f => generated 247 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:19:22.352032 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a6a674-1a4a-48d6-9d1a-1418953b28d3 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:22.430810 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 18/33] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:19:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 07:19:22.431651 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d09bcaf-3422-4d74-bf70-c9aa4cd452c5 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:22.476430 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a6a674-1a4a-48d6-9d1a-1418953b28d3 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:22.491744 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:22.521503 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76504328-e4f2-44c7-a9f9-8cf104819989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:22.522087 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76504328-e4f2-44c7-a9f9-8cf104819989 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:22.522488 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76504328-e4f2-44c7-a9f9-8cf104819989 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:22.522989 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76504328-e4f2-44c7-a9f9-8cf104819989 None admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:22.524189 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76504328-e4f2-44c7-a9f9-8cf104819989 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:19:22.524751 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76504328-e4f2-44c7-a9f9-8cf104819989 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:22.529168 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76504328-e4f2-44c7-a9f9-8cf104819989 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:22.545179 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d09bcaf-3422-4d74-bf70-c9aa4cd452c5 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:22.549999 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 16/34] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 15 07:19:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:19:22.550843 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76504328-e4f2-44c7-a9f9-8cf104819989 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:22.565121 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:22.585279 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b068d15b-a43c-4856-9a33-dba5ff40c9c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:22.586311 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b068d15b-a43c-4856-9a33-dba5ff40c9c7 None admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:22.586806 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b068d15b-a43c-4856-9a33-dba5ff40c9c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:22.587373 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b068d15b-a43c-4856-9a33-dba5ff40c9c7 None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:22.588346 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b068d15b-a43c-4856-9a33-dba5ff40c9c7 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:22.591022 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b068d15b-a43c-4856-9a33-dba5ff40c9c7 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:22.665757 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76504328-e4f2-44c7-a9f9-8cf104819989 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:22.706338 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f48f03f-3bc2-4882-88e5-60146fed2eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:22.706988 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f48f03f-3bc2-4882-88e5-60146fed2eb8 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:22.707272 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f48f03f-3bc2-4882-88e5-60146fed2eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:22.707550 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f48f03f-3bc2-4882-88e5-60146fed2eb8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:22.708771 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f48f03f-3bc2-4882-88e5-60146fed2eb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:22.710513 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f48f03f-3bc2-4882-88e5-60146fed2eb8 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:22.717697 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 17/35] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:19:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:19:22.718930 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b54961-5bc2-49e6-bf99-467150c115ee None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:22.721510 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b54961-5bc2-49e6-bf99-467150c115ee None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:22.721510 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b54961-5bc2-49e6-bf99-467150c115ee None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:22.721510 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 18/36] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:19:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:19:22.721962 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b54961-5bc2-49e6-bf99-467150c115ee None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:22.759754 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:22.821949 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 19/37] 10.222.0.42 () {60 vars in 1131 bytes} [Thu Sep 15 07:19:22 2022] POST /identity/v3/roles => generated 219 bytes in 391 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:19:22.823964 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b068d15b-a43c-4856-9a33-dba5ff40c9c7 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:22.858527 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b54961-5bc2-49e6-bf99-467150c115ee None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:22.930318 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bda720-e642-4d2b-927b-0f0d7082e5e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:22.931385 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bda720-e642-4d2b-927b-0f0d7082e5e7 None admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:22.931891 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bda720-e642-4d2b-927b-0f0d7082e5e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:22.932376 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bda720-e642-4d2b-927b-0f0d7082e5e7 None admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:22.933432 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bda720-e642-4d2b-927b-0f0d7082e5e7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:22.935916 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bda720-e642-4d2b-927b-0f0d7082e5e7 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:23.016984 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b068d15b-a43c-4856-9a33-dba5ff40c9c7 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:23.059484 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a847f6-008b-4586-a439-38e1cdb9467a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:23.060524 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a847f6-008b-4586-a439-38e1cdb9467a None admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:23.061030 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a847f6-008b-4586-a439-38e1cdb9467a None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:23.061454 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a847f6-008b-4586-a439-38e1cdb9467a None admin] PATH_INFO: `/v3/domains/default/users/06cea60dcdfb4c5ead6de8db2f0fd82f/roles/6bbb643e0d8d4bc0a9b27c26edf59216` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:23.063728 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a847f6-008b-4586-a439-38e1cdb9467a None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=06cea60dcdfb4c5ead6de8db2f0fd82f, role_id=6bbb643e0d8d4bc0a9b27c26edf59216)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:23.065322 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a847f6-008b-4586-a439-38e1cdb9467a None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:23.141999 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 20/38] 10.222.0.42 () {58 vars in 1361 bytes} [Thu Sep 15 07:19:22 2022] PUT /identity/v3/domains/default/users/06cea60dcdfb4c5ead6de8db2f0fd82f/roles/6bbb643e0d8d4bc0a9b27c26edf59216 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:19:23.155604 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afbcd11-a345-4b7b-80aa-5ee9f0da58af None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:23.155998 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afbcd11-a345-4b7b-80aa-5ee9f0da58af None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:23.156371 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afbcd11-a345-4b7b-80aa-5ee9f0da58af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:23.179029 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-5afbcd11-a345-4b7b-80aa-5ee9f0da58af None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:19:23.180925 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-5afbcd11-a345-4b7b-80aa-5ee9f0da58af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:19:23.191777 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 19/39] 10.222.0.42 () {60 vars in 1141 bytes} [Thu Sep 15 07:19:22 2022] POST /identity/v3/projects => generated 301 bytes in 470 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:19:23.202232 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bda720-e642-4d2b-927b-0f0d7082e5e7 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:23.225942 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:23.290731 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 21/40] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:19:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 07:19:23.293100 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c23521-1dfe-4337-8232-fd9b146c7075 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:23.293100 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c23521-1dfe-4337-8232-fd9b146c7075 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:23.293678 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c23521-1dfe-4337-8232-fd9b146c7075 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:23.297746 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 22/41] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:19:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:19:23.303991 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7415e19f-aa48-4a7e-bcc8-08b85ba57381 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:23.304294 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7415e19f-aa48-4a7e-bcc8-08b85ba57381 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:23.304606 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7415e19f-aa48-4a7e-bcc8-08b85ba57381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:23.358835 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-7415e19f-aa48-4a7e-bcc8-08b85ba57381 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:19:23.360067 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-7415e19f-aa48-4a7e-bcc8-08b85ba57381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:19:23.373716 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:23.392244 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bda720-e642-4d2b-927b-0f0d7082e5e7 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:23.437379 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080fc03a-4bb3-4bf3-9b68-6e00750cf058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:23.442977 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080fc03a-4bb3-4bf3-9b68-6e00750cf058 None admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:23.442977 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080fc03a-4bb3-4bf3-9b68-6e00750cf058 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:23.442977 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080fc03a-4bb3-4bf3-9b68-6e00750cf058 None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:23.442977 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080fc03a-4bb3-4bf3-9b68-6e00750cf058 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:23.442977 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080fc03a-4bb3-4bf3-9b68-6e00750cf058 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:23.450600 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 23/42] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:19:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 07:19:23.531137 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 20/43] 10.222.0.42 () {60 vars in 1131 bytes} [Thu Sep 15 07:19:23 2022] POST /identity/v3/roles => generated 225 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:19:23.595468 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7415e19f-aa48-4a7e-bcc8-08b85ba57381 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:23.706671 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:23.716732 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7415e19f-aa48-4a7e-bcc8-08b85ba57381 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:23.728500 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080fc03a-4bb3-4bf3-9b68-6e00750cf058 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:23.858603 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080fc03a-4bb3-4bf3-9b68-6e00750cf058 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:23.863091 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382bd694-3064-49df-8a9a-c0c17e29e1be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:23.863909 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382bd694-3064-49df-8a9a-c0c17e29e1be None admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:23.864336 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382bd694-3064-49df-8a9a-c0c17e29e1be None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:23.864753 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382bd694-3064-49df-8a9a-c0c17e29e1be None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:23.865718 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382bd694-3064-49df-8a9a-c0c17e29e1be None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:23.867108 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382bd694-3064-49df-8a9a-c0c17e29e1be None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:23.892930 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07c509c-ef10-48f4-8bba-6747c4c4f0aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:23.893567 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07c509c-ef10-48f4-8bba-6747c4c4f0aa None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:23.893879 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07c509c-ef10-48f4-8bba-6747c4c4f0aa None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:23.894155 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07c509c-ef10-48f4-8bba-6747c4c4f0aa None admin] PATH_INFO: `/v3/domains/default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:23.895348 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07c509c-ef10-48f4-8bba-6747c4c4f0aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:23.896786 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07c509c-ef10-48f4-8bba-6747c4c4f0aa None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:23.898401 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 21/44] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:19:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:19:23.904131 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07c509c-ef10-48f4-8bba-6747c4c4f0aa None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:23.920321 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:23.937004 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 24/45] 10.222.0.42 () {60 vars in 1131 bytes} [Thu Sep 15 07:19:23 2022] POST /identity/v3/roles => generated 223 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:19:24.023786 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07c509c-ef10-48f4-8bba-6747c4c4f0aa None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:24.061980 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388ce265-7406-426c-89f9-feb54caed80a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:24.062627 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388ce265-7406-426c-89f9-feb54caed80a None admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:24.062915 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388ce265-7406-426c-89f9-feb54caed80a None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:24.063195 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388ce265-7406-426c-89f9-feb54caed80a None admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:24.063881 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388ce265-7406-426c-89f9-feb54caed80a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:24.065375 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388ce265-7406-426c-89f9-feb54caed80a None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:24.142956 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 22/46] 10.222.0.42 () {60 vars in 1141 bytes} [Thu Sep 15 07:19:23 2022] POST /identity/v3/projects => generated 312 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 07:19:25.513312 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1404776d-aedc-422f-9462-18416b921648 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:25.513893 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1404776d-aedc-422f-9462-18416b921648 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:25.513893 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1404776d-aedc-422f-9462-18416b921648 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:25.515249 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 25/47] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:19:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:19:25.520650 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39a4048-ae01-4071-8aae-67acdade35a4 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:25.521063 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39a4048-ae01-4071-8aae-67acdade35a4 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:25.521214 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39a4048-ae01-4071-8aae-67acdade35a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:25.539799 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-b39a4048-ae01-4071-8aae-67acdade35a4 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:19:25.541295 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-b39a4048-ae01-4071-8aae-67acdade35a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:19:25.598844 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:25.605890 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 23/48] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:19:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 07:19:25.873672 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1404776d-aedc-422f-9462-18416b921648 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:25.977324 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1404776d-aedc-422f-9462-18416b921648 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:26.011567 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38346084-836b-4fe3-9f78-e9a79b311319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:26.012338 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38346084-836b-4fe3-9f78-e9a79b311319 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:26.012628 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38346084-836b-4fe3-9f78-e9a79b311319 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:26.012909 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38346084-836b-4fe3-9f78-e9a79b311319 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:26.022931 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38346084-836b-4fe3-9f78-e9a79b311319 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:19:26.028519 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-38346084-836b-4fe3-9f78-e9a79b311319 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 07:19:26.028519 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:19:26.028519 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:19:26.028519 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:19:26.028519 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:19:26.028519 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:19:26.028519 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:19:26.028519 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:19:26.028519 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:19:26.028519 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:19:26.028519 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:19:26.028519 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:19:26.028519 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:19:26.028519 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:19:26.028519 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:19:26.028519 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:19:26.029358 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:19:26.029358 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:19:26.029358 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:19:26.029358 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:19:26.029358 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:19:26.029358 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:19:26.029358 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:19:26.029358 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:19:26.029358 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:19:26.029358 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:19:26.029358 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:19:26.029358 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:19:26.029358 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:19:26.029358 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:19:26.029358 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:19:26.029358 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:19:26.030375 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:19:26.030375 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:19:26.030375 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:19:26.030375 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:19:26.030375 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:19:26.030375 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:19:26.030375 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:19:26.030375 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:19:26.030375 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:19:26.030375 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:19:26.030375 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 07:19:26.030375 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:19:26.030375 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 26/49] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 15 07:19:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:19:26.035434 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39a4048-ae01-4071-8aae-67acdade35a4 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:26.133525 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39a4048-ae01-4071-8aae-67acdade35a4 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:26.172058 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa944571-8c52-4aff-9ef0-3f350875ffce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:26.172681 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa944571-8c52-4aff-9ef0-3f350875ffce None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:26.172964 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa944571-8c52-4aff-9ef0-3f350875ffce None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:26.173239 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa944571-8c52-4aff-9ef0-3f350875ffce None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:26.173944 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa944571-8c52-4aff-9ef0-3f350875ffce None admin] RBAC: Adding query filter params (name=admin) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:19:26.174307 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa944571-8c52-4aff-9ef0-3f350875ffce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:26.175706 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa944571-8c52-4aff-9ef0-3f350875ffce None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:26.193023 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 24/50] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 15 07:19:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:19:26.199854 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38346084-836b-4fe3-9f78-e9a79b311319 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:26.220379 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:26.306396 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38346084-836b-4fe3-9f78-e9a79b311319 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:26.328004 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7a60cb-0c52-474b-b952-491253f78ecb None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:26.328305 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7a60cb-0c52-474b-b952-491253f78ecb None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:26.328596 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7a60cb-0c52-474b-b952-491253f78ecb None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:26.329984 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 25/51] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:19:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 07:19:26.334923 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9408b7-d236-428e-87a3-0950dd25bb68 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:26.335217 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9408b7-d236-428e-87a3-0950dd25bb68 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:26.335501 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9408b7-d236-428e-87a3-0950dd25bb68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:26.342024 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e91a691-cc38-4dcf-8271-c32cdee16dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:26.342659 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e91a691-cc38-4dcf-8271-c32cdee16dde None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:26.342937 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e91a691-cc38-4dcf-8271-c32cdee16dde None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:26.343215 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e91a691-cc38-4dcf-8271-c32cdee16dde None admin] PATH_INFO: `/v3/users/06cea60dcdfb4c5ead6de8db2f0fd82f` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:26.345868 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e91a691-cc38-4dcf-8271-c32cdee16dde None admin] RBAC: Authorizing `identity:get_user(user_id=06cea60dcdfb4c5ead6de8db2f0fd82f)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:26.345868 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e91a691-cc38-4dcf-8271-c32cdee16dde None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:26.347528 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 27/52] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 15 07:19:26 2022] GET /identity/v3/users/06cea60dcdfb4c5ead6de8db2f0fd82f => generated 247 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:19:26.353268 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e91a691-cc38-4dcf-8271-c32cdee16dde None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:26.447804 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:26.456791 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-fa9408b7-d236-428e-87a3-0950dd25bb68 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:19:26.458053 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-fa9408b7-d236-428e-87a3-0950dd25bb68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:19:26.465075 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e91a691-cc38-4dcf-8271-c32cdee16dde None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:26.500894 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c3154c-bc4d-41d7-9605-1cd1b98700e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:26.501534 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c3154c-bc4d-41d7-9605-1cd1b98700e0 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:26.501845 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c3154c-bc4d-41d7-9605-1cd1b98700e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:26.502121 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c3154c-bc4d-41d7-9605-1cd1b98700e0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:26.503245 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c3154c-bc4d-41d7-9605-1cd1b98700e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:26.504798 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c3154c-bc4d-41d7-9605-1cd1b98700e0 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:26.506442 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 28/53] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:19:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:19:26.513090 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c3154c-bc4d-41d7-9605-1cd1b98700e0 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:26.515411 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:26.529108 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 26/54] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:19:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 07:19:26.612349 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c3154c-bc4d-41d7-9605-1cd1b98700e0 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:26.646590 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e68473-ef75-4ad0-9566-0d7e5dcddcf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:26.647205 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e68473-ef75-4ad0-9566-0d7e5dcddcf0 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:26.647492 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e68473-ef75-4ad0-9566-0d7e5dcddcf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:26.647770 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e68473-ef75-4ad0-9566-0d7e5dcddcf0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:26.648492 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e68473-ef75-4ad0-9566-0d7e5dcddcf0 None admin] RBAC: Adding query filter params (role.id=6bbb643e0d8d4bc0a9b27c26edf59216, scope.domain.id=default, user.id=06cea60dcdfb4c5ead6de8db2f0fd82f) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:19:26.648812 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e68473-ef75-4ad0-9566-0d7e5dcddcf0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:26.650303 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e68473-ef75-4ad0-9566-0d7e5dcddcf0 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:26.656108 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 29/55] 10.222.0.42 () {56 vars in 1322 bytes} [Thu Sep 15 07:19:26 2022] GET /identity/v3/role_assignments?user.id=06cea60dcdfb4c5ead6de8db2f0fd82f&scope.domain.id=default&role.id=6bbb643e0d8d4bc0a9b27c26edf59216 => generated 530 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:19:26.801982 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9408b7-d236-428e-87a3-0950dd25bb68 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:26.903299 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9408b7-d236-428e-87a3-0950dd25bb68 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:26.936366 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349ce1ec-b0bc-4d9f-a7d7-c84d50fe0fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:26.937005 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349ce1ec-b0bc-4d9f-a7d7-c84d50fe0fe0 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:26.937292 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349ce1ec-b0bc-4d9f-a7d7-c84d50fe0fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:26.937571 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349ce1ec-b0bc-4d9f-a7d7-c84d50fe0fe0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:26.938720 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349ce1ec-b0bc-4d9f-a7d7-c84d50fe0fe0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:26.940167 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349ce1ec-b0bc-4d9f-a7d7-c84d50fe0fe0 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:26.941788 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 27/56] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:19:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:19:26.947366 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e68473-ef75-4ad0-9566-0d7e5dcddcf0 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:26.966811 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:27.047720 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e68473-ef75-4ad0-9566-0d7e5dcddcf0 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:27.080992 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eea47a-4640-404c-b9b9-9288cb3965e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:27.081540 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eea47a-4640-404c-b9b9-9288cb3965e5 None admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:27.081856 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eea47a-4640-404c-b9b9-9288cb3965e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:27.082137 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eea47a-4640-404c-b9b9-9288cb3965e5 None admin] PATH_INFO: `/v3/projects` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:27.082786 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6eea47a-4640-404c-b9b9-9288cb3965e5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:27.084236 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6eea47a-4640-404c-b9b9-9288cb3965e5 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:27.163910 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 30/57] 10.222.0.42 () {60 vars in 1140 bytes} [Thu Sep 15 07:19:26 2022] POST /identity/v3/projects => generated 298 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:19:29.366008 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88e21e4-da49-42ad-a329-264bd80eb0b6 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:29.366522 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88e21e4-da49-42ad-a329-264bd80eb0b6 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:29.366606 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88e21e4-da49-42ad-a329-264bd80eb0b6 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:29.367897 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 28/58] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:19:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:19:29.373701 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448dc507-aa9d-48c7-80fa-968733b3a7d6 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:29.374094 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448dc507-aa9d-48c7-80fa-968733b3a7d6 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:29.374261 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448dc507-aa9d-48c7-80fa-968733b3a7d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:29.392757 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-448dc507-aa9d-48c7-80fa-968733b3a7d6 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:19:29.393972 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-448dc507-aa9d-48c7-80fa-968733b3a7d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:19:29.453771 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:29.458876 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 31/59] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:19:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 07:19:29.723485 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88e21e4-da49-42ad-a329-264bd80eb0b6 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:29.779462 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:29.824066 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88e21e4-da49-42ad-a329-264bd80eb0b6 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:29.856033 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3d5722-23c1-4dda-a34b-5f8fba3e3a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:29.857317 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3d5722-23c1-4dda-a34b-5f8fba3e3a77 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:29.857624 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3d5722-23c1-4dda-a34b-5f8fba3e3a77 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:29.857904 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3d5722-23c1-4dda-a34b-5f8fba3e3a77 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:29.859016 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3d5722-23c1-4dda-a34b-5f8fba3e3a77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:29.860445 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3d5722-23c1-4dda-a34b-5f8fba3e3a77 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:29.862024 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 29/60] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:19:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:19:29.867465 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448dc507-aa9d-48c7-80fa-968733b3a7d6 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:29.962788 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448dc507-aa9d-48c7-80fa-968733b3a7d6 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:29.996596 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf401784-8cd5-4226-9073-c92808c698c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:29.997231 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf401784-8cd5-4226-9073-c92808c698c0 None admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:29.997512 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf401784-8cd5-4226-9073-c92808c698c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:29.997817 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf401784-8cd5-4226-9073-c92808c698c0 None admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:29.998473 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf401784-8cd5-4226-9073-c92808c698c0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:29.999923 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf401784-8cd5-4226-9073-c92808c698c0 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:30.105878 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 32/61] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Sep 15 07:19:29 2022] POST /identity/v3/users => generated 275 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:19:33.098397 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ce5e27-5f96-4735-b37a-a29958d54ddd None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:33.098918 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ce5e27-5f96-4735-b37a-a29958d54ddd None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:33.098992 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ce5e27-5f96-4735-b37a-a29958d54ddd None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:33.100357 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 30/62] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:19:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:19:33.115622 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a869f58-c3c7-4655-bd2b-c29e6f38576e None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:33.116094 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a869f58-c3c7-4655-bd2b-c29e6f38576e None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:33.116271 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a869f58-c3c7-4655-bd2b-c29e6f38576e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:33.142626 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-9a869f58-c3c7-4655-bd2b-c29e6f38576e None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:19:33.143872 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-9a869f58-c3c7-4655-bd2b-c29e6f38576e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:19:33.208028 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:33.235497 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 33/63] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:19:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 07:19:33.387939 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4099b9-e281-450c-a27e-f3985504b6c9 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:33.388400 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4099b9-e281-450c-a27e-f3985504b6c9 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:33.388731 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4099b9-e281-450c-a27e-f3985504b6c9 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:33.390373 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 31/64] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:19:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:19:33.395740 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfa0638-c71e-4468-b424-d11af84c84a0 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:33.396042 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfa0638-c71e-4468-b424-d11af84c84a0 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:33.396328 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfa0638-c71e-4468-b424-d11af84c84a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:33.422643 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-bcfa0638-c71e-4468-b424-d11af84c84a0 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:19:33.423856 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-bcfa0638-c71e-4468-b424-d11af84c84a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:19:33.519934 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 34/65] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:19:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 07:19:33.709900 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fce0551-23c3-4bab-8005-eab767e5a117 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:33.710270 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fce0551-23c3-4bab-8005-eab767e5a117 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:33.710470 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fce0551-23c3-4bab-8005-eab767e5a117 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:33.712092 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 32/66] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:19:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:19:33.717198 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ddd3af-d447-45ba-966e-17ee0bedf7b6 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:33.717752 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ddd3af-d447-45ba-966e-17ee0bedf7b6 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:33.718255 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ddd3af-d447-45ba-966e-17ee0bedf7b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:33.746599 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-66ddd3af-d447-45ba-966e-17ee0bedf7b6 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:19:33.748683 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-66ddd3af-d447-45ba-966e-17ee0bedf7b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:19:33.817376 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fce0551-23c3-4bab-8005-eab767e5a117 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:33.846623 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 35/67] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:19:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 07:19:33.881124 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ddd3af-d447-45ba-966e-17ee0bedf7b6 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:33.966511 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:34.020714 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fce0551-23c3-4bab-8005-eab767e5a117 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:34.071202 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:34.077285 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ddd3af-d447-45ba-966e-17ee0bedf7b6 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:34.095829 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2407aa1-c4b4-4227-8473-a938d7dfcbc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:34.096484 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2407aa1-c4b4-4227-8473-a938d7dfcbc9 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:34.096768 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2407aa1-c4b4-4227-8473-a938d7dfcbc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:34.097043 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2407aa1-c4b4-4227-8473-a938d7dfcbc9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:34.124725 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2407aa1-c4b4-4227-8473-a938d7dfcbc9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:19:34.133412 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-d2407aa1-c4b4-4227-8473-a938d7dfcbc9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 07:19:34.133412 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:19:34.133412 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:19:34.133412 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:19:34.133412 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:19:34.133412 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:19:34.133412 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:19:34.133412 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:19:34.133412 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:19:34.133412 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:19:34.133412 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:19:34.133412 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:19:34.133412 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:19:34.133412 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:19:34.133412 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:19:34.133412 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:19:34.134299 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:19:34.134299 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:19:34.134299 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:19:34.134299 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:19:34.134299 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:19:34.134299 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:19:34.134299 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:19:34.134299 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:19:34.134299 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:19:34.134299 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:19:34.134299 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:19:34.134299 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:19:34.134299 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:19:34.134299 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:19:34.134299 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:19:34.134299 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:19:34.135062 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:19:34.135062 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:19:34.135062 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:19:34.135062 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:19:34.135062 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:19:34.135062 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:19:34.135062 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:19:34.135062 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:19:34.135062 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:19:34.135062 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:19:34.135062 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 07:19:34.135062 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:19:34.135627 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6822fa2-0e4b-4a5a-8101-58a96750efba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:34.135627 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6822fa2-0e4b-4a5a-8101-58a96750efba None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:34.135627 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6822fa2-0e4b-4a5a-8101-58a96750efba None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:34.135627 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6822fa2-0e4b-4a5a-8101-58a96750efba None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:34.138978 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 33/68] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 15 07:19:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 07:19:34.140082 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2407aa1-c4b4-4227-8473-a938d7dfcbc9 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:34.148249 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6822fa2-0e4b-4a5a-8101-58a96750efba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:19:34.154332 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-c6822fa2-0e4b-4a5a-8101-58a96750efba None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 07:19:34.154332 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:19:34.154332 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:19:34.154332 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:19:34.154332 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:19:34.154332 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:19:34.154332 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:19:34.154332 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:19:34.154332 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:19:34.154332 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:19:34.154332 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:19:34.154332 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:19:34.154332 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:19:34.154332 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:19:34.154332 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:19:34.154332 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:19:34.155225 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:19:34.155225 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:19:34.155225 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:19:34.155225 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:19:34.155225 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:19:34.155225 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:19:34.155225 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:19:34.155225 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:19:34.155225 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:19:34.155225 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:19:34.155225 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:19:34.155225 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:19:34.155225 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:19:34.155225 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:19:34.155225 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:19:34.155225 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:19:34.155949 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:19:34.155949 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:19:34.155949 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:19:34.155949 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:19:34.155949 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:19:34.155949 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:19:34.155949 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:19:34.155949 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:19:34.155949 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:19:34.155949 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:19:34.155949 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 07:19:34.155949 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:19:34.155949 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 36/69] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 15 07:19:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 07:19:34.157288 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c531685-7a60-4e2d-aa5c-860abdb1b132 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:34.157384 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c531685-7a60-4e2d-aa5c-860abdb1b132 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:34.157685 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c531685-7a60-4e2d-aa5c-860abdb1b132 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:34.158890 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 37/70] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:19:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:19:34.159591 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c531685-7a60-4e2d-aa5c-860abdb1b132 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:34.259712 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2407aa1-c4b4-4227-8473-a938d7dfcbc9 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:34.274154 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c531685-7a60-4e2d-aa5c-860abdb1b132 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:34.277221 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:34.298888 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2e3ee2-3355-471b-8917-c889913d6b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:34.299556 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2e3ee2-3355-471b-8917-c889913d6b10 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:34.299833 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2e3ee2-3355-471b-8917-c889913d6b10 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:34.300111 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2e3ee2-3355-471b-8917-c889913d6b10 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:34.311934 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2e3ee2-3355-471b-8917-c889913d6b10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:19:34.314575 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5508b2b6-5993-446f-a272-b549a68105d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:34.315191 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5508b2b6-5993-446f-a272-b549a68105d2 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:34.315473 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5508b2b6-5993-446f-a272-b549a68105d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:34.315748 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5508b2b6-5993-446f-a272-b549a68105d2 None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:34.316404 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5508b2b6-5993-446f-a272-b549a68105d2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:19:34.316713 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5508b2b6-5993-446f-a272-b549a68105d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:34.316861 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-7a2e3ee2-3355-471b-8917-c889913d6b10 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 07:19:34.316861 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:19:34.316861 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:19:34.316861 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:19:34.316861 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:19:34.316861 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:19:34.316861 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:19:34.316861 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:19:34.316861 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:19:34.316861 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:19:34.316861 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:19:34.316861 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:19:34.316861 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:19:34.316861 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:19:34.316861 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:19:34.316861 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:19:34.317899 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:19:34.317899 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:19:34.317899 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:19:34.317899 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:19:34.317899 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:19:34.317899 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:19:34.317899 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:19:34.317899 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:19:34.317899 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:19:34.317899 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:19:34.317899 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:19:34.317899 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:19:34.317899 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:19:34.317899 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:19:34.317899 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:19:34.317899 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:19:34.318870 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:19:34.318870 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:19:34.318870 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:19:34.318870 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:19:34.318870 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:19:34.318870 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:19:34.318870 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:19:34.318870 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:19:34.318870 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:19:34.318870 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:19:34.318870 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 07:19:34.318870 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:19:34.318870 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 34/71] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Sep 15 07:19:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:19:34.319708 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5508b2b6-5993-446f-a272-b549a68105d2 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:34.322464 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2e3ee2-3355-471b-8917-c889913d6b10 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:34.327770 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 38/72] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Sep 15 07:19:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:19:34.329292 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f19f2e3-6fda-443b-afc7-f3c3f284a09a None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:34.329604 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f19f2e3-6fda-443b-afc7-f3c3f284a09a None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:34.329898 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f19f2e3-6fda-443b-afc7-f3c3f284a09a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:34.347605 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-9f19f2e3-6fda-443b-afc7-f3c3f284a09a None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:19:34.348767 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-9f19f2e3-6fda-443b-afc7-f3c3f284a09a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:19:34.368033 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:34.416638 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 39/73] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:19:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 07:19:34.418183 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414269f9-5e4d-4099-8441-acf59043227a None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:34.418480 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414269f9-5e4d-4099-8441-acf59043227a None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:34.419341 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414269f9-5e4d-4099-8441-acf59043227a None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:34.420742 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 40/74] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:19:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:19:34.421446 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414269f9-5e4d-4099-8441-acf59043227a None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:34.432489 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2e3ee2-3355-471b-8917-c889913d6b10 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:34.468531 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a92dd3-a630-4eef-8236-746f2f456856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:34.469364 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a92dd3-a630-4eef-8236-746f2f456856 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:34.469672 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a92dd3-a630-4eef-8236-746f2f456856 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:34.469952 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a92dd3-a630-4eef-8236-746f2f456856 None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:34.470614 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a92dd3-a630-4eef-8236-746f2f456856 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:19:34.470919 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a92dd3-a630-4eef-8236-746f2f456856 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:34.472301 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a92dd3-a630-4eef-8236-746f2f456856 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:34.480326 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 35/75] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 15 07:19:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:19:34.486541 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a92dd3-a630-4eef-8236-746f2f456856 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:34.534123 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:34.541426 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414269f9-5e4d-4099-8441-acf59043227a None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:34.580175 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad70662-e720-40c8-a34a-d82a48d93079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:34.581115 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad70662-e720-40c8-a34a-d82a48d93079 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:34.581403 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad70662-e720-40c8-a34a-d82a48d93079 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:34.581853 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad70662-e720-40c8-a34a-d82a48d93079 None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:34.582524 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad70662-e720-40c8-a34a-d82a48d93079 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:19:34.582830 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad70662-e720-40c8-a34a-d82a48d93079 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:34.584259 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad70662-e720-40c8-a34a-d82a48d93079 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:34.598028 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 41/76] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 15 07:19:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:19:34.598028 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385e9dee-bc21-4bf6-844b-3722d16ea94b None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:34.598028 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385e9dee-bc21-4bf6-844b-3722d16ea94b None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:34.598028 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385e9dee-bc21-4bf6-844b-3722d16ea94b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:34.601865 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a92dd3-a630-4eef-8236-746f2f456856 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:34.609258 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:34.613464 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-385e9dee-bc21-4bf6-844b-3722d16ea94b None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:19:34.614694 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-385e9dee-bc21-4bf6-844b-3722d16ea94b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:19:34.639483 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1cdd39-e9fc-43ea-b376-215f73cfac06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:34.640130 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1cdd39-e9fc-43ea-b376-215f73cfac06 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:34.640412 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1cdd39-e9fc-43ea-b376-215f73cfac06 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:34.640692 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1cdd39-e9fc-43ea-b376-215f73cfac06 None admin] PATH_INFO: `/v3/users/001806a30226487ab9e17f8d988f1ca6` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:34.656788 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1cdd39-e9fc-43ea-b376-215f73cfac06 None admin] RBAC: Authorizing `identity:get_user(user_id=001806a30226487ab9e17f8d988f1ca6)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:34.658265 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1cdd39-e9fc-43ea-b376-215f73cfac06 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:34.664253 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 36/77] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 15 07:19:34 2022] GET /identity/v3/users/001806a30226487ab9e17f8d988f1ca6 => generated 275 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:19:34.664253 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1cdd39-e9fc-43ea-b376-215f73cfac06 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:34.690991 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 42/78] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:19:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 07:19:34.691847 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385e9dee-bc21-4bf6-844b-3722d16ea94b None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:34.771750 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1cdd39-e9fc-43ea-b376-215f73cfac06 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:34.782645 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:34.795079 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385e9dee-bc21-4bf6-844b-3722d16ea94b None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:34.808659 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8affe2aa-71d1-4f8d-8dd9-50b294224d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:34.809296 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8affe2aa-71d1-4f8d-8dd9-50b294224d04 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:34.809638 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8affe2aa-71d1-4f8d-8dd9-50b294224d04 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:34.809927 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8affe2aa-71d1-4f8d-8dd9-50b294224d04 None admin] PATH_INFO: `/v3/users/06cea60dcdfb4c5ead6de8db2f0fd82f` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:34.811076 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8affe2aa-71d1-4f8d-8dd9-50b294224d04 None admin] RBAC: Authorizing `identity:get_user(user_id=06cea60dcdfb4c5ead6de8db2f0fd82f)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:34.812536 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8affe2aa-71d1-4f8d-8dd9-50b294224d04 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:34.818593 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 37/79] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 15 07:19:34 2022] GET /identity/v3/users/06cea60dcdfb4c5ead6de8db2f0fd82f => generated 247 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:19:34.818593 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8affe2aa-71d1-4f8d-8dd9-50b294224d04 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:34.832534 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ab6671-3bb5-4335-b75d-a8c7d4d4ad69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:34.833129 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ab6671-3bb5-4335-b75d-a8c7d4d4ad69 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:34.833418 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ab6671-3bb5-4335-b75d-a8c7d4d4ad69 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:34.833732 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ab6671-3bb5-4335-b75d-a8c7d4d4ad69 None admin] PATH_INFO: `/v3/users/001806a30226487ab9e17f8d988f1ca6` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:34.834885 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ab6671-3bb5-4335-b75d-a8c7d4d4ad69 None admin] RBAC: Authorizing `identity:get_user(user_id=001806a30226487ab9e17f8d988f1ca6)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:34.836472 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ab6671-3bb5-4335-b75d-a8c7d4d4ad69 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:34.838266 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 43/80] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 15 07:19:34 2022] GET /identity/v3/users/001806a30226487ab9e17f8d988f1ca6 => generated 275 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:19:34.842791 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ab6671-3bb5-4335-b75d-a8c7d4d4ad69 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:34.852818 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:34.936028 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8affe2aa-71d1-4f8d-8dd9-50b294224d04 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:34.953461 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ab6671-3bb5-4335-b75d-a8c7d4d4ad69 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:34.975932 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca13791-1411-406e-881a-da78f9f0da99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:34.976539 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca13791-1411-406e-881a-da78f9f0da99 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:34.976825 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca13791-1411-406e-881a-da78f9f0da99 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:34.977105 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca13791-1411-406e-881a-da78f9f0da99 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:34.989931 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9b6a77-2648-484e-8a09-012a3e03c34f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:34.990565 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9b6a77-2648-484e-8a09-012a3e03c34f None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:34.990847 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9b6a77-2648-484e-8a09-012a3e03c34f None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:34.991120 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9b6a77-2648-484e-8a09-012a3e03c34f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:35.001186 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9b6a77-2648-484e-8a09-012a3e03c34f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:19:35.006051 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-9e9b6a77-2648-484e-8a09-012a3e03c34f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 07:19:35.006051 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:19:35.006051 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:19:35.006051 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:19:35.006051 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:19:35.006051 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:19:35.006051 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:19:35.006051 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:19:35.006051 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:19:35.006051 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:19:35.006051 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:19:35.006051 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:19:35.006051 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:19:35.006051 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:19:35.006051 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:19:35.006051 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:19:35.006912 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:19:35.006912 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:19:35.006912 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:19:35.006912 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:19:35.006912 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:19:35.006912 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:19:35.006912 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:19:35.006912 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:19:35.006912 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:19:35.006912 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:19:35.006912 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:19:35.006912 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:19:35.006912 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:19:35.006912 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:19:35.006912 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:19:35.006912 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:19:35.007663 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:19:35.007663 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:19:35.007663 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:19:35.007663 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:19:35.007663 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:19:35.007663 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:19:35.007663 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:19:35.007663 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:19:35.007663 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:19:35.007663 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:19:35.007663 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 07:19:35.007663 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:19:35.007663 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 44/81] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 15 07:19:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:19:35.009341 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9b6a77-2648-484e-8a09-012a3e03c34f None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:35.028221 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:35.047213 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca13791-1411-406e-881a-da78f9f0da99 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:35.048751 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca13791-1411-406e-881a-da78f9f0da99 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:35.080049 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:35.109427 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9b6a77-2648-484e-8a09-012a3e03c34f None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:35.144199 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d93e962-3765-4699-9705-7db40d82c418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:35.144813 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d93e962-3765-4699-9705-7db40d82c418 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:35.145079 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d93e962-3765-4699-9705-7db40d82c418 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:35.145356 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d93e962-3765-4699-9705-7db40d82c418 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:35.148422 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 38/82] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 15 07:19:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 07:19:35.149316 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca13791-1411-406e-881a-da78f9f0da99 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:35.217228 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d93e962-3765-4699-9705-7db40d82c418 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:35.218718 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d93e962-3765-4699-9705-7db40d82c418 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:35.237070 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:35.251109 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca13791-1411-406e-881a-da78f9f0da99 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:35.280429 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:35.285046 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c586d6e7-4b8a-4f75-8692-403993bea84c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:35.285691 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c586d6e7-4b8a-4f75-8692-403993bea84c None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:35.285973 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c586d6e7-4b8a-4f75-8692-403993bea84c None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:35.286245 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c586d6e7-4b8a-4f75-8692-403993bea84c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:35.319969 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 45/83] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 15 07:19:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 07:19:35.320823 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d93e962-3765-4699-9705-7db40d82c418 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:35.353933 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c586d6e7-4b8a-4f75-8692-403993bea84c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:35.355378 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c586d6e7-4b8a-4f75-8692-403993bea84c None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:35.425436 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d93e962-3765-4699-9705-7db40d82c418 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:35.441979 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:35.451615 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 39/84] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 15 07:19:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 07:19:35.452287 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c586d6e7-4b8a-4f75-8692-403993bea84c None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:35.459934 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e96b55e-bd50-4fdf-80ba-3f016f859220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:35.460640 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e96b55e-bd50-4fdf-80ba-3f016f859220 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:35.460923 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e96b55e-bd50-4fdf-80ba-3f016f859220 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:35.461200 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e96b55e-bd50-4fdf-80ba-3f016f859220 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:35.462452 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e96b55e-bd50-4fdf-80ba-3f016f859220 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:35.463916 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e96b55e-bd50-4fdf-80ba-3f016f859220 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:35.465512 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 46/85] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:19:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:19:35.465512 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e96b55e-bd50-4fdf-80ba-3f016f859220 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:35.484268 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:35.552894 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c586d6e7-4b8a-4f75-8692-403993bea84c None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:35.568128 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e96b55e-bd50-4fdf-80ba-3f016f859220 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:35.587654 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13212fd0-319c-4d3b-ada3-62cec174e7bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:35.588270 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13212fd0-319c-4d3b-ada3-62cec174e7bd None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:35.588565 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13212fd0-319c-4d3b-ada3-62cec174e7bd None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:35.588842 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13212fd0-319c-4d3b-ada3-62cec174e7bd None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:35.589506 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13212fd0-319c-4d3b-ada3-62cec174e7bd None admin] RBAC: Adding query filter params (name=member) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:19:35.589849 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13212fd0-319c-4d3b-ada3-62cec174e7bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:35.591323 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13212fd0-319c-4d3b-ada3-62cec174e7bd None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:35.599449 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 40/86] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Sep 15 07:19:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:19:35.600244 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13212fd0-319c-4d3b-ada3-62cec174e7bd None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:35.605121 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d250cfe-c0d4-436c-95d0-5367eff4529a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:35.605788 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d250cfe-c0d4-436c-95d0-5367eff4529a None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:35.606073 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d250cfe-c0d4-436c-95d0-5367eff4529a None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:35.607021 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d250cfe-c0d4-436c-95d0-5367eff4529a None admin] PATH_INFO: `/v3/projects/659ee9038adf47e585b94881867c18c2` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:35.608151 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d250cfe-c0d4-436c-95d0-5367eff4529a None admin] RBAC: Authorizing `identity:get_project(project_id=659ee9038adf47e585b94881867c18c2)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:35.609641 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d250cfe-c0d4-436c-95d0-5367eff4529a None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:35.611267 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 47/87] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 15 07:19:35 2022] GET /identity/v3/projects/659ee9038adf47e585b94881867c18c2 => generated 298 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:19:35.612024 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d250cfe-c0d4-436c-95d0-5367eff4529a None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:35.663845 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:35.699139 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:35.706737 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13212fd0-319c-4d3b-ada3-62cec174e7bd None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:35.711040 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d250cfe-c0d4-436c-95d0-5367eff4529a None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:35.740760 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a045c0b-445c-4c74-a1ea-c58c35d88f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:35.741389 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a045c0b-445c-4c74-a1ea-c58c35d88f06 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:35.741705 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a045c0b-445c-4c74-a1ea-c58c35d88f06 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:35.741985 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a045c0b-445c-4c74-a1ea-c58c35d88f06 None admin] PATH_INFO: `/v3/projects/659ee9038adf47e585b94881867c18c2` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:35.743074 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a045c0b-445c-4c74-a1ea-c58c35d88f06 None admin] RBAC: Authorizing `identity:get_project(project_id=659ee9038adf47e585b94881867c18c2)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:35.743478 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45467c4-b89d-4e78-b2ee-99f9ca950384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:35.744105 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45467c4-b89d-4e78-b2ee-99f9ca950384 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:35.744387 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45467c4-b89d-4e78-b2ee-99f9ca950384 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:35.744509 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a045c0b-445c-4c74-a1ea-c58c35d88f06 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:35.744686 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45467c4-b89d-4e78-b2ee-99f9ca950384 None admin] PATH_INFO: `/v3/projects/659ee9038adf47e585b94881867c18c2` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:35.745792 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45467c4-b89d-4e78-b2ee-99f9ca950384 None admin] RBAC: Authorizing `identity:get_project(project_id=659ee9038adf47e585b94881867c18c2)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:35.746333 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 41/88] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 15 07:19:35 2022] GET /identity/v3/projects/659ee9038adf47e585b94881867c18c2 => generated 298 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:19:35.747053 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a045c0b-445c-4c74-a1ea-c58c35d88f06 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:35.747449 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45467c4-b89d-4e78-b2ee-99f9ca950384 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:35.749199 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 48/89] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 15 07:19:35 2022] GET /identity/v3/projects/659ee9038adf47e585b94881867c18c2 => generated 298 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:19:35.749852 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45467c4-b89d-4e78-b2ee-99f9ca950384 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:35.847341 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a045c0b-445c-4c74-a1ea-c58c35d88f06 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:35.849677 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45467c4-b89d-4e78-b2ee-99f9ca950384 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:35.869779 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:35.881203 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4dae2f-5a8a-4b58-baf2-fb4917b063da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:35.881839 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4dae2f-5a8a-4b58-baf2-fb4917b063da None admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:35.882126 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4dae2f-5a8a-4b58-baf2-fb4917b063da None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:35.882400 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4dae2f-5a8a-4b58-baf2-fb4917b063da None admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:35.884618 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4dae2f-5a8a-4b58-baf2-fb4917b063da None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:35.884737 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2a6840-4ef0-40f9-9652-b672a3cbffec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:35.884737 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2a6840-4ef0-40f9-9652-b672a3cbffec None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:35.884737 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2a6840-4ef0-40f9-9652-b672a3cbffec None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:35.884996 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4dae2f-5a8a-4b58-baf2-fb4917b063da None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:35.885086 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2a6840-4ef0-40f9-9652-b672a3cbffec None admin] PATH_INFO: `/v3/users/001806a30226487ab9e17f8d988f1ca6` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:35.885732 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2a6840-4ef0-40f9-9652-b672a3cbffec None admin] RBAC: Authorizing `identity:get_user(user_id=001806a30226487ab9e17f8d988f1ca6)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:35.887174 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2a6840-4ef0-40f9-9652-b672a3cbffec None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:35.889509 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 49/90] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 15 07:19:35 2022] GET /identity/v3/users/001806a30226487ab9e17f8d988f1ca6 => generated 275 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:19:35.890294 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2a6840-4ef0-40f9-9652-b672a3cbffec None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:35.987539 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 42/91] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Sep 15 07:19:35 2022] POST /identity/v3/users => generated 289 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:19:35.988421 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4dae2f-5a8a-4b58-baf2-fb4917b063da None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:35.998677 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2a6840-4ef0-40f9-9652-b672a3cbffec None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:36.012756 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:36.038139 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d46b357-4806-4b80-9add-53d7722b6fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:36.038767 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d46b357-4806-4b80-9add-53d7722b6fc1 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:36.039054 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d46b357-4806-4b80-9add-53d7722b6fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:36.039331 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d46b357-4806-4b80-9add-53d7722b6fc1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:36.040082 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d46b357-4806-4b80-9add-53d7722b6fc1 None admin] RBAC: Adding query filter params (role.id=053e12af84e943e699460a9d76c2072b, scope.project.id=659ee9038adf47e585b94881867c18c2, user.id=001806a30226487ab9e17f8d988f1ca6) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:19:36.040395 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d46b357-4806-4b80-9add-53d7722b6fc1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:36.041925 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d46b357-4806-4b80-9add-53d7722b6fc1 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:36.061618 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 50/92] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 15 07:19:35 2022] GET /identity/v3/role_assignments?user.id=001806a30226487ab9e17f8d988f1ca6&scope.project.id=659ee9038adf47e585b94881867c18c2&role.id=053e12af84e943e699460a9d76c2072b => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:19:36.061618 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d46b357-4806-4b80-9add-53d7722b6fc1 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:36.127058 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4dae2f-5a8a-4b58-baf2-fb4917b063da None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:36.142901 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:36.169299 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8f620d-7da4-497e-8ee0-211051bb00fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:36.169299 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8f620d-7da4-497e-8ee0-211051bb00fe None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:36.170699 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8f620d-7da4-497e-8ee0-211051bb00fe None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:36.170699 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8f620d-7da4-497e-8ee0-211051bb00fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:36.171443 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8f620d-7da4-497e-8ee0-211051bb00fe None admin] RBAC: Adding query filter params (role.id=6bbb643e0d8d4bc0a9b27c26edf59216, scope.project.id=659ee9038adf47e585b94881867c18c2, user.id=06cea60dcdfb4c5ead6de8db2f0fd82f) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:19:36.171845 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8f620d-7da4-497e-8ee0-211051bb00fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:36.173373 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8f620d-7da4-497e-8ee0-211051bb00fe None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:36.185395 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 43/93] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 15 07:19:35 2022] GET /identity/v3/role_assignments?user.id=06cea60dcdfb4c5ead6de8db2f0fd82f&scope.project.id=659ee9038adf47e585b94881867c18c2&role.id=6bbb643e0d8d4bc0a9b27c26edf59216 => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:19:36.185969 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8f620d-7da4-497e-8ee0-211051bb00fe None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:36.220239 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d46b357-4806-4b80-9add-53d7722b6fc1 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:36.275464 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20346e2-d45d-485f-9eee-5d49818e2d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:36.276317 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20346e2-d45d-485f-9eee-5d49818e2d6b None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:36.276603 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20346e2-d45d-485f-9eee-5d49818e2d6b None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:36.276883 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20346e2-d45d-485f-9eee-5d49818e2d6b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:36.277863 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20346e2-d45d-485f-9eee-5d49818e2d6b None admin] RBAC: Adding query filter params (role.id=a0fae926490542b4977d1ecf27fcece4, scope.project.id=659ee9038adf47e585b94881867c18c2, user.id=001806a30226487ab9e17f8d988f1ca6) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:19:36.278745 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20346e2-d45d-485f-9eee-5d49818e2d6b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:36.280336 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20346e2-d45d-485f-9eee-5d49818e2d6b None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:36.288035 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 51/94] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 15 07:19:36 2022] GET /identity/v3/role_assignments?user.id=001806a30226487ab9e17f8d988f1ca6&scope.project.id=659ee9038adf47e585b94881867c18c2&role.id=a0fae926490542b4977d1ecf27fcece4 => generated 259 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:19:36.330294 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:36.354856 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8f620d-7da4-497e-8ee0-211051bb00fe None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:36.396727 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d24d4a-d5af-47df-b42e-1e0f3fedeaad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:36.397362 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d24d4a-d5af-47df-b42e-1e0f3fedeaad None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:36.397680 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d24d4a-d5af-47df-b42e-1e0f3fedeaad None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:36.397958 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d24d4a-d5af-47df-b42e-1e0f3fedeaad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:36.480754 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d24d4a-d5af-47df-b42e-1e0f3fedeaad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:36.483419 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d24d4a-d5af-47df-b42e-1e0f3fedeaad None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:36.580789 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:36.601083 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 44/95] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 15 07:19:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 07:19:36.607039 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20346e2-d45d-485f-9eee-5d49818e2d6b None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:36.726316 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20346e2-d45d-485f-9eee-5d49818e2d6b None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:36.746409 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:36.766022 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f30269-fd13-4317-b7e2-5107ec2f2094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:36.766848 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f30269-fd13-4317-b7e2-5107ec2f2094 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:36.767239 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f30269-fd13-4317-b7e2-5107ec2f2094 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:36.767765 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f30269-fd13-4317-b7e2-5107ec2f2094 None admin] PATH_INFO: `/v3/projects/adc861e34dc44f389899da5a1c6f0770` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:36.769076 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f30269-fd13-4317-b7e2-5107ec2f2094 None admin] RBAC: Authorizing `identity:get_project(project_id=adc861e34dc44f389899da5a1c6f0770)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:36.770687 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f30269-fd13-4317-b7e2-5107ec2f2094 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:36.772780 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 52/96] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 15 07:19:36 2022] GET /identity/v3/projects/adc861e34dc44f389899da5a1c6f0770 => generated 312 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:19:36.779924 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d24d4a-d5af-47df-b42e-1e0f3fedeaad None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:36.889809 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d24d4a-d5af-47df-b42e-1e0f3fedeaad None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:36.926802 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a6c034-a108-4c9f-842e-5e623add6fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:36.927430 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a6c034-a108-4c9f-842e-5e623add6fbc None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:36.927712 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a6c034-a108-4c9f-842e-5e623add6fbc None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:36.927989 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a6c034-a108-4c9f-842e-5e623add6fbc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:36.928727 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a6c034-a108-4c9f-842e-5e623add6fbc None admin] RBAC: Adding query filter params (role.id=053e12af84e943e699460a9d76c2072b, scope.project.id=adc861e34dc44f389899da5a1c6f0770, user.id=001806a30226487ab9e17f8d988f1ca6) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:19:36.929043 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a6c034-a108-4c9f-842e-5e623add6fbc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:36.930869 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a6c034-a108-4c9f-842e-5e623add6fbc None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:36.937940 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 45/97] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 15 07:19:36 2022] GET /identity/v3/role_assignments?user.id=001806a30226487ab9e17f8d988f1ca6&scope.project.id=adc861e34dc44f389899da5a1c6f0770&role.id=053e12af84e943e699460a9d76c2072b => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:19:38.850685 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6296f8-57b9-4ee2-bf02-87ff2d30f6b5 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:38.851192 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6296f8-57b9-4ee2-bf02-87ff2d30f6b5 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:38.851264 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6296f8-57b9-4ee2-bf02-87ff2d30f6b5 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:38.852736 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 53/98] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:19:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:19:38.864165 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1802e2-4cc5-40ff-8b46-f6411e1ba653 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:38.864606 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1802e2-4cc5-40ff-8b46-f6411e1ba653 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:38.864782 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1802e2-4cc5-40ff-8b46-f6411e1ba653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:38.889666 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-5a1802e2-4cc5-40ff-8b46-f6411e1ba653 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:19:38.890956 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-5a1802e2-4cc5-40ff-8b46-f6411e1ba653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:19:38.952707 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:38.978465 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 46/99] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:19:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 07:19:39.262250 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768c5c36-6267-4b3c-ad4d-4f9c7df661aa None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:39.262614 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768c5c36-6267-4b3c-ad4d-4f9c7df661aa None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:39.262855 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768c5c36-6267-4b3c-ad4d-4f9c7df661aa None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:39.264826 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 54/100] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:19:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:19:39.283482 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3415f1a-383d-4fb8-87d2-4c722ba4787d None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:39.283768 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3415f1a-383d-4fb8-87d2-4c722ba4787d None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:39.284058 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3415f1a-383d-4fb8-87d2-4c722ba4787d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:39.303358 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-c3415f1a-383d-4fb8-87d2-4c722ba4787d None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:19:39.304822 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-c3415f1a-383d-4fb8-87d2-4c722ba4787d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:19:39.381157 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 47/101] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:19:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 07:19:39.432388 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768c5c36-6267-4b3c-ad4d-4f9c7df661aa None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:39.577311 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768c5c36-6267-4b3c-ad4d-4f9c7df661aa None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:39.623921 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2705b64b-8fc6-4ffd-915e-f395188462e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:39.624581 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2705b64b-8fc6-4ffd-915e-f395188462e6 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:39.624867 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2705b64b-8fc6-4ffd-915e-f395188462e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:39.625146 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2705b64b-8fc6-4ffd-915e-f395188462e6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:39.646497 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2705b64b-8fc6-4ffd-915e-f395188462e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:19:39.651859 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-2705b64b-8fc6-4ffd-915e-f395188462e6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 07:19:39.651859 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:19:39.651859 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:19:39.651859 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:19:39.651859 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:19:39.651859 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:19:39.651859 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:19:39.651859 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:19:39.651859 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:19:39.651859 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:19:39.651859 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:19:39.651859 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:19:39.651859 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:19:39.651859 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:19:39.651859 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:19:39.651859 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:19:39.652730 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:19:39.652730 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:19:39.652730 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:19:39.652730 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:19:39.652730 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:19:39.652730 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:19:39.652730 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:19:39.652730 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:19:39.652730 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:19:39.652730 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:19:39.652730 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:19:39.652730 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:19:39.652730 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:19:39.652730 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:19:39.652730 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:19:39.652730 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:19:39.653588 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:19:39.653588 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:19:39.653588 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:19:39.653588 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:19:39.653588 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:19:39.653588 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:19:39.653588 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:19:39.653588 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:19:39.653588 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:19:39.653588 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:19:39.653588 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 07:19:39.653588 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:19:39.653588 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 55/102] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 15 07:19:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:19:39.658639 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfe0ffc-5236-4c2d-a19d-eeeb706e3e86 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:39.658639 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfe0ffc-5236-4c2d-a19d-eeeb706e3e86 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:39.658639 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfe0ffc-5236-4c2d-a19d-eeeb706e3e86 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:39.659675 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 48/103] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:19:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:19:39.664936 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfe0ffc-5236-4c2d-a19d-eeeb706e3e86 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:39.665076 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dafd0d7-f52f-4c79-a38b-e839c674d47f None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:39.665271 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dafd0d7-f52f-4c79-a38b-e839c674d47f None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:39.665548 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dafd0d7-f52f-4c79-a38b-e839c674d47f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:39.702290 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-7dafd0d7-f52f-4c79-a38b-e839c674d47f None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:19:39.703915 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-7dafd0d7-f52f-4c79-a38b-e839c674d47f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:19:39.777928 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:39.838791 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 56/104] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:19:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 07:19:39.839475 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dafd0d7-f52f-4c79-a38b-e839c674d47f None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:39.877053 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfe0ffc-5236-4c2d-a19d-eeeb706e3e86 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:39.885998 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:39.951021 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c154c869-f129-4e22-a2a8-de7c10144336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:39.951682 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c154c869-f129-4e22-a2a8-de7c10144336 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:39.951977 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c154c869-f129-4e22-a2a8-de7c10144336 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:39.952276 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c154c869-f129-4e22-a2a8-de7c10144336 None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:39.953066 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c154c869-f129-4e22-a2a8-de7c10144336 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:19:39.953385 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c154c869-f129-4e22-a2a8-de7c10144336 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:39.954922 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c154c869-f129-4e22-a2a8-de7c10144336 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:39.969039 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 49/105] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 15 07:19:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:19:39.970434 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620438b8-a079-4a59-b297-3e380c5b983d None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:39.970731 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620438b8-a079-4a59-b297-3e380c5b983d None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:39.971017 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620438b8-a079-4a59-b297-3e380c5b983d None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:39.977630 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 50/106] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:19:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:19:39.978466 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620438b8-a079-4a59-b297-3e380c5b983d None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:40.026751 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dafd0d7-f52f-4c79-a38b-e839c674d47f None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:40.101821 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a535ad1-a10f-455d-b4d7-646124b4fa92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:40.103636 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a535ad1-a10f-455d-b4d7-646124b4fa92 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:40.103917 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a535ad1-a10f-455d-b4d7-646124b4fa92 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:40.104193 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a535ad1-a10f-455d-b4d7-646124b4fa92 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:40.132654 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a535ad1-a10f-455d-b4d7-646124b4fa92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:19:40.138552 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-6a535ad1-a10f-455d-b4d7-646124b4fa92 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 07:19:40.138552 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:19:40.138552 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:19:40.138552 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:19:40.138552 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:19:40.138552 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:19:40.138552 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:19:40.138552 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:19:40.138552 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:19:40.138552 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:19:40.138552 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:19:40.138552 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:19:40.138552 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:19:40.138552 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:19:40.138552 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:19:40.138552 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:19:40.139418 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:19:40.139418 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:19:40.139418 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:19:40.139418 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:19:40.139418 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:19:40.139418 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:19:40.139418 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:19:40.139418 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:19:40.139418 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:19:40.139418 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:19:40.139418 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:19:40.139418 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:19:40.139418 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:19:40.139418 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:19:40.139418 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:19:40.139418 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:19:40.140170 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:19:40.140170 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:19:40.140170 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:19:40.140170 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:19:40.140170 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:19:40.140170 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:19:40.140170 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:19:40.140170 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:19:40.140170 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:19:40.140170 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:19:40.140170 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 07:19:40.140170 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:19:40.142300 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 57/107] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 15 07:19:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:19:40.144198 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba708945-f508-4fb8-9f7c-81388b9b6198 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:40.144660 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba708945-f508-4fb8-9f7c-81388b9b6198 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:40.145106 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba708945-f508-4fb8-9f7c-81388b9b6198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:40.163189 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-ba708945-f508-4fb8-9f7c-81388b9b6198 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:19:40.164375 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-ba708945-f508-4fb8-9f7c-81388b9b6198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:19:40.164483 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620438b8-a079-4a59-b297-3e380c5b983d None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:40.196681 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:40.209048 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bddf8a-c11f-4a46-861e-9aceba6a0f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:40.209759 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bddf8a-c11f-4a46-861e-9aceba6a0f73 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:40.210135 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bddf8a-c11f-4a46-861e-9aceba6a0f73 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:40.210413 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bddf8a-c11f-4a46-861e-9aceba6a0f73 None admin] PATH_INFO: `/v3/users/06cea60dcdfb4c5ead6de8db2f0fd82f` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:40.211562 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bddf8a-c11f-4a46-861e-9aceba6a0f73 None admin] RBAC: Authorizing `identity:get_user(user_id=06cea60dcdfb4c5ead6de8db2f0fd82f)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:40.213081 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bddf8a-c11f-4a46-861e-9aceba6a0f73 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:40.219606 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 51/108] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 15 07:19:39 2022] GET /identity/v3/users/06cea60dcdfb4c5ead6de8db2f0fd82f => generated 247 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:19:40.219606 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508f370e-2ae2-4c28-9f8e-19843adc120d None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:40.219606 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508f370e-2ae2-4c28-9f8e-19843adc120d None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:40.219606 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508f370e-2ae2-4c28-9f8e-19843adc120d None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:40.219606 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 52/109] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:19:40 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:19:40.219606 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508f370e-2ae2-4c28-9f8e-19843adc120d None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:40.257611 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 58/110] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:19:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 07:19:40.257611 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba708945-f508-4fb8-9f7c-81388b9b6198 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:40.268849 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:40.345812 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508f370e-2ae2-4c28-9f8e-19843adc120d None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:40.383291 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aa696b-fe89-4123-b1af-00f1dd16b650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:40.383930 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa696b-fe89-4123-b1af-00f1dd16b650 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:40.384280 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa696b-fe89-4123-b1af-00f1dd16b650 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:40.384631 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa696b-fe89-4123-b1af-00f1dd16b650 None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:40.385373 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aa696b-fe89-4123-b1af-00f1dd16b650 None admin] RBAC: Adding query filter params (name=member) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:19:40.385825 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aa696b-fe89-4123-b1af-00f1dd16b650 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:40.387339 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aa696b-fe89-4123-b1af-00f1dd16b650 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:40.391882 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba708945-f508-4fb8-9f7c-81388b9b6198 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:40.398248 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 53/111] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Sep 15 07:19:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:19:40.399076 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aa696b-fe89-4123-b1af-00f1dd16b650 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:40.434939 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cfc4db-ca5d-4b39-97ee-748ab2ab3507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:40.436415 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cfc4db-ca5d-4b39-97ee-748ab2ab3507 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:40.436709 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cfc4db-ca5d-4b39-97ee-748ab2ab3507 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:40.436992 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cfc4db-ca5d-4b39-97ee-748ab2ab3507 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:40.438178 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cfc4db-ca5d-4b39-97ee-748ab2ab3507 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:40.440179 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cfc4db-ca5d-4b39-97ee-748ab2ab3507 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:40.442418 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 59/112] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:19:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:19:40.443607 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbec8d4-a749-45b1-8c82-0e0a73f649e3 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:40.443901 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbec8d4-a749-45b1-8c82-0e0a73f649e3 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:40.444240 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbec8d4-a749-45b1-8c82-0e0a73f649e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:40.463510 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-bdbec8d4-a749-45b1-8c82-0e0a73f649e3 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:19:40.464694 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-bdbec8d4-a749-45b1-8c82-0e0a73f649e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:19:40.494183 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:40.505077 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:40.508886 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aa696b-fe89-4123-b1af-00f1dd16b650 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:40.538005 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 60/113] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:19:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 07:19:40.542940 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a750252-75c4-4618-886e-c892c935778c None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:40.543230 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a750252-75c4-4618-886e-c892c935778c None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:40.543509 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a750252-75c4-4618-886e-c892c935778c None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:40.544448 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01009350-dc91-4593-a29f-819c915c55e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:40.545042 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 61/114] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:19:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:19:40.545741 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a750252-75c4-4618-886e-c892c935778c None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:40.546106 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01009350-dc91-4593-a29f-819c915c55e6 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:40.546606 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01009350-dc91-4593-a29f-819c915c55e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:40.547084 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01009350-dc91-4593-a29f-819c915c55e6 None admin] PATH_INFO: `/v3/projects/659ee9038adf47e585b94881867c18c2` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:40.548643 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01009350-dc91-4593-a29f-819c915c55e6 None admin] RBAC: Authorizing `identity:get_project(project_id=659ee9038adf47e585b94881867c18c2)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:40.551142 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01009350-dc91-4593-a29f-819c915c55e6 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:40.552991 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 54/115] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 15 07:19:40 2022] GET /identity/v3/projects/659ee9038adf47e585b94881867c18c2 => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:19:40.558518 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01009350-dc91-4593-a29f-819c915c55e6 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:40.663619 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a750252-75c4-4618-886e-c892c935778c None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:40.667639 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01009350-dc91-4593-a29f-819c915c55e6 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:40.703384 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132eaa70-7bd7-4e85-a277-190e25972819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:40.704004 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132eaa70-7bd7-4e85-a277-190e25972819 None admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:40.704295 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132eaa70-7bd7-4e85-a277-190e25972819 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:40.704576 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132eaa70-7bd7-4e85-a277-190e25972819 None admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:40.705231 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132eaa70-7bd7-4e85-a277-190e25972819 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:40.706743 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132eaa70-7bd7-4e85-a277-190e25972819 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:40.707501 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd44b27-c6d4-483c-b571-88b20de4bcf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:40.708134 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd44b27-c6d4-483c-b571-88b20de4bcf4 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:40.708412 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd44b27-c6d4-483c-b571-88b20de4bcf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:40.708684 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd44b27-c6d4-483c-b571-88b20de4bcf4 None admin] PATH_INFO: `/v3/users/001806a30226487ab9e17f8d988f1ca6` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:40.709841 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd44b27-c6d4-483c-b571-88b20de4bcf4 None admin] RBAC: Authorizing `identity:get_user(user_id=001806a30226487ab9e17f8d988f1ca6)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:40.711278 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd44b27-c6d4-483c-b571-88b20de4bcf4 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:40.712870 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 62/116] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 15 07:19:40 2022] GET /identity/v3/users/001806a30226487ab9e17f8d988f1ca6 => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:19:40.718487 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd44b27-c6d4-483c-b571-88b20de4bcf4 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:40.748726 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:40.806116 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 55/117] 10.222.0.42 () {60 vars in 1141 bytes} [Thu Sep 15 07:19:40 2022] POST /identity/v3/projects => generated 302 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:19:40.807412 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1e31bc-3b84-40bb-b774-e3c7364a3946 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:40.807698 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1e31bc-3b84-40bb-b774-e3c7364a3946 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:40.807972 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1e31bc-3b84-40bb-b774-e3c7364a3946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:40.822950 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd44b27-c6d4-483c-b571-88b20de4bcf4 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:40.840606 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-2b1e31bc-3b84-40bb-b774-e3c7364a3946 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:19:40.842955 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-2b1e31bc-3b84-40bb-b774-e3c7364a3946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:19:40.863192 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:40.863381 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68632b75-6026-4814-a5bb-a2050da1710c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:40.864099 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68632b75-6026-4814-a5bb-a2050da1710c None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:40.864305 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68632b75-6026-4814-a5bb-a2050da1710c None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:40.864597 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68632b75-6026-4814-a5bb-a2050da1710c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:40.877258 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68632b75-6026-4814-a5bb-a2050da1710c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:19:40.882412 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-68632b75-6026-4814-a5bb-a2050da1710c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 07:19:40.882412 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:19:40.882412 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:19:40.882412 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:19:40.882412 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:19:40.882412 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:19:40.882412 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:19:40.882412 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:19:40.882412 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:19:40.882412 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:19:40.882412 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:19:40.882412 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:19:40.882412 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:19:40.882412 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:19:40.882412 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:19:40.882412 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:19:40.883322 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:19:40.883322 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:19:40.883322 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:19:40.883322 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:19:40.883322 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:19:40.883322 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:19:40.883322 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:19:40.883322 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:19:40.883322 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:19:40.883322 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:19:40.883322 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:19:40.883322 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:19:40.883322 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:19:40.883322 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:19:40.883322 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:19:40.883322 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:19:40.884069 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:19:40.884069 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:19:40.884069 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:19:40.884069 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:19:40.884069 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:19:40.884069 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:19:40.884069 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:19:40.884069 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:19:40.884069 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:19:40.884069 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:19:40.884069 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 07:19:40.884069 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:19:40.885966 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 63/118] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 15 07:19:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:19:40.887378 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68632b75-6026-4814-a5bb-a2050da1710c None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:40.928029 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 56/119] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:19:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 07:19:40.939268 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1e31bc-3b84-40bb-b774-e3c7364a3946 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:41.007563 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:41.020862 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68632b75-6026-4814-a5bb-a2050da1710c None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:41.065293 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295e7eab-f022-46ec-b311-9c1c1fbf83ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:41.066074 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e7eab-f022-46ec-b311-9c1c1fbf83ba None admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:41.066364 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e7eab-f022-46ec-b311-9c1c1fbf83ba None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:41.066650 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e7eab-f022-46ec-b311-9c1c1fbf83ba None admin] PATH_INFO: `/v3/projects/659ee9038adf47e585b94881867c18c2/users/06cea60dcdfb4c5ead6de8db2f0fd82f/roles/6bbb643e0d8d4bc0a9b27c26edf59216` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:41.068671 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295e7eab-f022-46ec-b311-9c1c1fbf83ba None admin] RBAC: Authorizing `identity:create_grant(project_id=659ee9038adf47e585b94881867c18c2, user_id=06cea60dcdfb4c5ead6de8db2f0fd82f, role_id=6bbb643e0d8d4bc0a9b27c26edf59216)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:41.070175 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295e7eab-f022-46ec-b311-9c1c1fbf83ba None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:41.070744 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1e31bc-3b84-40bb-b774-e3c7364a3946 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:41.108423 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4a5138-bb84-42f0-a711-32055c8ad1f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:41.109065 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4a5138-bb84-42f0-a711-32055c8ad1f6 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:41.109349 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4a5138-bb84-42f0-a711-32055c8ad1f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:41.109680 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4a5138-bb84-42f0-a711-32055c8ad1f6 None admin] PATH_INFO: `/v3/projects/659ee9038adf47e585b94881867c18c2` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:41.110795 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4a5138-bb84-42f0-a711-32055c8ad1f6 None admin] RBAC: Authorizing `identity:get_project(project_id=659ee9038adf47e585b94881867c18c2)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:41.112226 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4a5138-bb84-42f0-a711-32055c8ad1f6 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:41.113915 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 57/120] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 15 07:19:40 2022] GET /identity/v3/projects/659ee9038adf47e585b94881867c18c2 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:19:41.114787 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4a5138-bb84-42f0-a711-32055c8ad1f6 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:41.134327 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:41.149756 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 64/121] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Sep 15 07:19:40 2022] PUT /identity/v3/projects/659ee9038adf47e585b94881867c18c2/users/06cea60dcdfb4c5ead6de8db2f0fd82f/roles/6bbb643e0d8d4bc0a9b27c26edf59216 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:19:41.150982 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295e7eab-f022-46ec-b311-9c1c1fbf83ba None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:41.280718 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4a5138-bb84-42f0-a711-32055c8ad1f6 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:41.292820 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295e7eab-f022-46ec-b311-9c1c1fbf83ba None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:41.330844 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b853fe-4341-4f6c-98a7-1c5e0801b270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:41.332100 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b853fe-4341-4f6c-98a7-1c5e0801b270 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:41.332477 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b853fe-4341-4f6c-98a7-1c5e0801b270 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:41.332981 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b853fe-4341-4f6c-98a7-1c5e0801b270 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:41.337399 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d20a446-37f8-44c0-ac78-fa8527d31c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:41.338313 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d20a446-37f8-44c0-ac78-fa8527d31c48 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:41.338772 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d20a446-37f8-44c0-ac78-fa8527d31c48 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:41.339222 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d20a446-37f8-44c0-ac78-fa8527d31c48 None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:41.340462 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d20a446-37f8-44c0-ac78-fa8527d31c48 None admin] RBAC: Adding query filter params (name=member) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:19:41.341016 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d20a446-37f8-44c0-ac78-fa8527d31c48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:41.342887 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d20a446-37f8-44c0-ac78-fa8527d31c48 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:41.346351 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b853fe-4341-4f6c-98a7-1c5e0801b270 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:19:41.351670 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-50b853fe-4341-4f6c-98a7-1c5e0801b270 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 07:19:41.351670 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:19:41.351670 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:19:41.351670 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:19:41.351670 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:19:41.351670 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:19:41.351670 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:19:41.351670 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:19:41.351670 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:19:41.351670 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:19:41.351670 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:19:41.351670 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:19:41.351670 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:19:41.351670 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:19:41.351670 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:19:41.351670 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:19:41.352545 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:19:41.352545 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:19:41.352545 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:19:41.352545 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:19:41.352545 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:19:41.352545 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:19:41.352545 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:19:41.352545 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:19:41.352545 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:19:41.352545 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:19:41.352545 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:19:41.352545 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:19:41.352545 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:19:41.352545 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:19:41.352545 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:19:41.352545 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:19:41.353279 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:19:41.353279 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:19:41.353279 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:19:41.353279 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:19:41.353279 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:19:41.353279 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:19:41.353279 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:19:41.353279 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:19:41.353279 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:19:41.353279 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:19:41.353279 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 07:19:41.353279 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:19:41.353279 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 58/122] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Sep 15 07:19:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:19:41.354198 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b853fe-4341-4f6c-98a7-1c5e0801b270 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:41.355731 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 65/123] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Sep 15 07:19:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:19:41.357349 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d20a446-37f8-44c0-ac78-fa8527d31c48 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:41.377047 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:41.458261 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:41.484337 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d20a446-37f8-44c0-ac78-fa8527d31c48 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:41.485605 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b853fe-4341-4f6c-98a7-1c5e0801b270 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:41.521836 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335ad5b6-fffb-4826-9738-9fb171da23c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:41.522687 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335ad5b6-fffb-4826-9738-9fb171da23c4 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:41.523038 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335ad5b6-fffb-4826-9738-9fb171da23c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:41.523381 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335ad5b6-fffb-4826-9738-9fb171da23c4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:41.527286 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7f02e3-4ecd-4e5b-aeb5-02e24ecb3b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:41.527962 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7f02e3-4ecd-4e5b-aeb5-02e24ecb3b8f None admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:41.529230 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7f02e3-4ecd-4e5b-aeb5-02e24ecb3b8f None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:41.529747 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7f02e3-4ecd-4e5b-aeb5-02e24ecb3b8f None admin] PATH_INFO: `/v3/projects/659ee9038adf47e585b94881867c18c2/users/001806a30226487ab9e17f8d988f1ca6/roles/053e12af84e943e699460a9d76c2072b` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:41.531905 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7f02e3-4ecd-4e5b-aeb5-02e24ecb3b8f None admin] RBAC: Authorizing `identity:create_grant(project_id=659ee9038adf47e585b94881867c18c2, user_id=001806a30226487ab9e17f8d988f1ca6, role_id=053e12af84e943e699460a9d76c2072b)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:41.533314 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7f02e3-4ecd-4e5b-aeb5-02e24ecb3b8f None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:41.535283 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335ad5b6-fffb-4826-9738-9fb171da23c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:19:41.541967 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-335ad5b6-fffb-4826-9738-9fb171da23c4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 07:19:41.541967 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:19:41.541967 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:19:41.541967 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:19:41.541967 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:19:41.541967 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:19:41.541967 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:19:41.541967 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:19:41.541967 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:19:41.541967 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:19:41.541967 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:19:41.541967 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:19:41.541967 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:19:41.541967 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:19:41.541967 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:19:41.541967 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:19:41.542902 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:19:41.542902 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:19:41.542902 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:19:41.542902 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:19:41.542902 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:19:41.542902 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:19:41.542902 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:19:41.542902 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:19:41.542902 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:19:41.542902 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:19:41.542902 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:19:41.542902 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:19:41.542902 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:19:41.542902 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:19:41.542902 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:19:41.542902 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:19:41.543678 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:19:41.543678 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:19:41.543678 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:19:41.543678 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:19:41.543678 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:19:41.543678 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:19:41.543678 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:19:41.543678 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:19:41.543678 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:19:41.543678 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:19:41.543678 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 07:19:41.543678 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:19:41.548810 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 66/124] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 15 07:19:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:19:41.549705 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335ad5b6-fffb-4826-9738-9fb171da23c4 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:41.618907 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 59/125] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Sep 15 07:19:41 2022] PUT /identity/v3/projects/659ee9038adf47e585b94881867c18c2/users/001806a30226487ab9e17f8d988f1ca6/roles/053e12af84e943e699460a9d76c2072b => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:19:41.619993 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7f02e3-4ecd-4e5b-aeb5-02e24ecb3b8f None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:41.674192 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:41.725829 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335ad5b6-fffb-4826-9738-9fb171da23c4 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:41.767557 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad505ce-d386-413f-9f53-a289f58a1ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:41.768212 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad505ce-d386-413f-9f53-a289f58a1ca7 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:41.768501 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad505ce-d386-413f-9f53-a289f58a1ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:41.768782 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad505ce-d386-413f-9f53-a289f58a1ca7 None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:41.769711 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad505ce-d386-413f-9f53-a289f58a1ca7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:19:41.770027 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad505ce-d386-413f-9f53-a289f58a1ca7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:41.771562 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad505ce-d386-413f-9f53-a289f58a1ca7 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:41.782994 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 67/126] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 15 07:19:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:19:41.783645 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad505ce-d386-413f-9f53-a289f58a1ca7 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:41.913053 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad505ce-d386-413f-9f53-a289f58a1ca7 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:41.961604 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7f02e3-4ecd-4e5b-aeb5-02e24ecb3b8f None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:41.993825 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5961e1-1993-49f2-a7b6-b2c9061399e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:41.994659 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5961e1-1993-49f2-a7b6-b2c9061399e0 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:41.994956 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5961e1-1993-49f2-a7b6-b2c9061399e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:41.995239 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5961e1-1993-49f2-a7b6-b2c9061399e0 None admin] PATH_INFO: `/v3/users/001806a30226487ab9e17f8d988f1ca6` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:41.996375 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5961e1-1993-49f2-a7b6-b2c9061399e0 None admin] RBAC: Authorizing `identity:get_user(user_id=001806a30226487ab9e17f8d988f1ca6)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:41.998705 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5961e1-1993-49f2-a7b6-b2c9061399e0 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:42.000487 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 60/127] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 15 07:19:41 2022] GET /identity/v3/users/001806a30226487ab9e17f8d988f1ca6 => generated 275 bytes in 381 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:19:42.001208 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5961e1-1993-49f2-a7b6-b2c9061399e0 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:42.005909 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7ccec6-b4a4-49e9-a0ef-957dcab0fe74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:42.006541 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7ccec6-b4a4-49e9-a0ef-957dcab0fe74 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:42.006826 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7ccec6-b4a4-49e9-a0ef-957dcab0fe74 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:42.007103 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7ccec6-b4a4-49e9-a0ef-957dcab0fe74 None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:42.007782 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7ccec6-b4a4-49e9-a0ef-957dcab0fe74 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:19:42.008088 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7ccec6-b4a4-49e9-a0ef-957dcab0fe74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:42.009470 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7ccec6-b4a4-49e9-a0ef-957dcab0fe74 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:42.010585 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:42.011697 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:42.030377 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 68/128] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Sep 15 07:19:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:19:42.031338 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7ccec6-b4a4-49e9-a0ef-957dcab0fe74 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:42.177524 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5961e1-1993-49f2-a7b6-b2c9061399e0 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:42.217661 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7ccec6-b4a4-49e9-a0ef-957dcab0fe74 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:42.247884 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82ad93e-beb7-427d-a9ef-77c3f833c41f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:42.248541 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82ad93e-beb7-427d-a9ef-77c3f833c41f None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:42.248824 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82ad93e-beb7-427d-a9ef-77c3f833c41f None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:42.249100 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82ad93e-beb7-427d-a9ef-77c3f833c41f None admin] PATH_INFO: `/v3/users/001806a30226487ab9e17f8d988f1ca6` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:42.250410 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82ad93e-beb7-427d-a9ef-77c3f833c41f None admin] RBAC: Authorizing `identity:get_user(user_id=001806a30226487ab9e17f8d988f1ca6)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:42.251853 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82ad93e-beb7-427d-a9ef-77c3f833c41f None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:42.253459 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 61/129] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 15 07:19:42 2022] GET /identity/v3/users/001806a30226487ab9e17f8d988f1ca6 => generated 275 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:19:42.258546 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82ad93e-beb7-427d-a9ef-77c3f833c41f None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:42.288767 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a221df-d5e7-4a67-8ab5-20f6dc29b0fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:42.289412 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a221df-d5e7-4a67-8ab5-20f6dc29b0fc None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:42.289721 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a221df-d5e7-4a67-8ab5-20f6dc29b0fc None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:42.289999 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a221df-d5e7-4a67-8ab5-20f6dc29b0fc None admin] PATH_INFO: `/v3/projects/adc861e34dc44f389899da5a1c6f0770` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:42.291240 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a221df-d5e7-4a67-8ab5-20f6dc29b0fc None admin] RBAC: Authorizing `identity:get_project(project_id=adc861e34dc44f389899da5a1c6f0770)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:42.292698 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a221df-d5e7-4a67-8ab5-20f6dc29b0fc None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:42.294755 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 69/130] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 15 07:19:42 2022] GET /identity/v3/projects/adc861e34dc44f389899da5a1c6f0770 => generated 312 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:19:42.296782 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a221df-d5e7-4a67-8ab5-20f6dc29b0fc None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:42.405950 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:42.412141 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:42.448564 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82ad93e-beb7-427d-a9ef-77c3f833c41f None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:42.473525 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a221df-d5e7-4a67-8ab5-20f6dc29b0fc None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:42.513199 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7cbd3a-b59f-4d00-ae14-f4aeb23b08cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:42.514087 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7cbd3a-b59f-4d00-ae14-f4aeb23b08cd None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:42.514372 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7cbd3a-b59f-4d00-ae14-f4aeb23b08cd None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:42.514644 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7cbd3a-b59f-4d00-ae14-f4aeb23b08cd None admin] PATH_INFO: `/v3/users/7fdb3deb1979491fa3690c7fdf6cc63b` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:42.536951 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859e8480-4154-49af-89fa-1537f249350c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:42.537603 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859e8480-4154-49af-89fa-1537f249350c None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:42.537896 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859e8480-4154-49af-89fa-1537f249350c None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:42.538173 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859e8480-4154-49af-89fa-1537f249350c None admin] PATH_INFO: `/v3/projects/659ee9038adf47e585b94881867c18c2` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:42.539222 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7cbd3a-b59f-4d00-ae14-f4aeb23b08cd None admin] RBAC: Authorizing `identity:get_user(user_id=7fdb3deb1979491fa3690c7fdf6cc63b)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:42.540761 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7cbd3a-b59f-4d00-ae14-f4aeb23b08cd None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:42.542633 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 62/131] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 15 07:19:42 2022] GET /identity/v3/users/7fdb3deb1979491fa3690c7fdf6cc63b => generated 289 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:19:42.548174 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859e8480-4154-49af-89fa-1537f249350c None admin] RBAC: Authorizing `identity:get_project(project_id=659ee9038adf47e585b94881867c18c2)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:42.549752 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859e8480-4154-49af-89fa-1537f249350c None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:42.551557 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 70/132] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 15 07:19:42 2022] GET /identity/v3/projects/659ee9038adf47e585b94881867c18c2 => generated 298 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:19:42.552243 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7cbd3a-b59f-4d00-ae14-f4aeb23b08cd None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:42.554196 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859e8480-4154-49af-89fa-1537f249350c None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:42.737278 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7cbd3a-b59f-4d00-ae14-f4aeb23b08cd None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:42.765948 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859e8480-4154-49af-89fa-1537f249350c None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:42.793985 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:42.806293 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ccfff4-9e23-48c8-afd8-a14460dcbc94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:42.806970 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ccfff4-9e23-48c8-afd8-a14460dcbc94 None admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:42.807345 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ccfff4-9e23-48c8-afd8-a14460dcbc94 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:42.807626 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ccfff4-9e23-48c8-afd8-a14460dcbc94 None admin] PATH_INFO: `/v3/projects/adc861e34dc44f389899da5a1c6f0770/users/001806a30226487ab9e17f8d988f1ca6/roles/053e12af84e943e699460a9d76c2072b` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:42.812064 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ccfff4-9e23-48c8-afd8-a14460dcbc94 None admin] RBAC: Authorizing `identity:create_grant(project_id=adc861e34dc44f389899da5a1c6f0770, user_id=001806a30226487ab9e17f8d988f1ca6, role_id=053e12af84e943e699460a9d76c2072b)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:42.813539 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ccfff4-9e23-48c8-afd8-a14460dcbc94 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:42.826369 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:42.831620 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dabd77-0928-4a52-b9ec-189a5d30f25f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:42.832293 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dabd77-0928-4a52-b9ec-189a5d30f25f None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:42.838917 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dabd77-0928-4a52-b9ec-189a5d30f25f None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:42.838917 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dabd77-0928-4a52-b9ec-189a5d30f25f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:42.890295 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 63/133] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Sep 15 07:19:42 2022] PUT /identity/v3/projects/adc861e34dc44f389899da5a1c6f0770/users/001806a30226487ab9e17f8d988f1ca6/roles/053e12af84e943e699460a9d76c2072b => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:19:42.891208 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ccfff4-9e23-48c8-afd8-a14460dcbc94 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:42.936391 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dabd77-0928-4a52-b9ec-189a5d30f25f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:42.940905 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dabd77-0928-4a52-b9ec-189a5d30f25f None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:43.152777 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ccfff4-9e23-48c8-afd8-a14460dcbc94 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:43.166821 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 71/134] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 15 07:19:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 613 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 07:19:43.173688 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dabd77-0928-4a52-b9ec-189a5d30f25f None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:43.234552 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:43.239651 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e908fd8a-a974-4968-85ef-56766addb00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:43.240342 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e908fd8a-a974-4968-85ef-56766addb00a None admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:43.240629 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e908fd8a-a974-4968-85ef-56766addb00a None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:43.240910 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e908fd8a-a974-4968-85ef-56766addb00a None admin] PATH_INFO: `/v3/projects/659ee9038adf47e585b94881867c18c2/users/001806a30226487ab9e17f8d988f1ca6/roles/a0fae926490542b4977d1ecf27fcece4` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:43.243157 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e908fd8a-a974-4968-85ef-56766addb00a None admin] RBAC: Authorizing `identity:create_grant(project_id=659ee9038adf47e585b94881867c18c2, user_id=001806a30226487ab9e17f8d988f1ca6, role_id=a0fae926490542b4977d1ecf27fcece4)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:43.244625 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e908fd8a-a974-4968-85ef-56766addb00a None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:43.302038 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 64/135] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Sep 15 07:19:42 2022] PUT /identity/v3/projects/659ee9038adf47e585b94881867c18c2/users/001806a30226487ab9e17f8d988f1ca6/roles/a0fae926490542b4977d1ecf27fcece4 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:19:43.303667 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20044cf5-7b03-4f8f-89a5-11caa649022e None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:43.303963 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20044cf5-7b03-4f8f-89a5-11caa649022e None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:43.304245 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20044cf5-7b03-4f8f-89a5-11caa649022e None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:43.306239 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 65/136] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:19:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:19:43.312872 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589881ec-fb5f-49a8-87d6-51e51b97f6db None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:43.313192 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589881ec-fb5f-49a8-87d6-51e51b97f6db None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:43.313462 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589881ec-fb5f-49a8-87d6-51e51b97f6db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:43.339319 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-589881ec-fb5f-49a8-87d6-51e51b97f6db None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:19:43.341333 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-589881ec-fb5f-49a8-87d6-51e51b97f6db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:19:43.348349 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dabd77-0928-4a52-b9ec-189a5d30f25f None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:43.406081 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:43.422080 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2652ab2a-df01-43df-9cc7-f53a14820159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:43.422690 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2652ab2a-df01-43df-9cc7-f53a14820159 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:43.422973 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2652ab2a-df01-43df-9cc7-f53a14820159 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:43.423251 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2652ab2a-df01-43df-9cc7-f53a14820159 None admin] PATH_INFO: `/v3/projects/659ee9038adf47e585b94881867c18c2` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:43.424470 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2652ab2a-df01-43df-9cc7-f53a14820159 None admin] RBAC: Authorizing `identity:get_project(project_id=659ee9038adf47e585b94881867c18c2)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:43.426274 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2652ab2a-df01-43df-9cc7-f53a14820159 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:43.428016 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 72/137] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 15 07:19:43 2022] GET /identity/v3/projects/659ee9038adf47e585b94881867c18c2 => generated 298 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:19:43.428798 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 66/138] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:19:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 07:19:43.434426 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2652ab2a-df01-43df-9cc7-f53a14820159 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:43.556832 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:43.560670 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2652ab2a-df01-43df-9cc7-f53a14820159 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:43.601848 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeab3cb-ac5e-4f96-b2f0-0f3a389916ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:43.602534 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeab3cb-ac5e-4f96-b2f0-0f3a389916ec None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:43.604066 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeab3cb-ac5e-4f96-b2f0-0f3a389916ec None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:43.604350 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeab3cb-ac5e-4f96-b2f0-0f3a389916ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:43.605146 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeab3cb-ac5e-4f96-b2f0-0f3a389916ec None admin] RBAC: Adding query filter params (role.id=81b8a5256ba14184beb300947fbfe335, scope.project.id=659ee9038adf47e585b94881867c18c2, user.id=7fdb3deb1979491fa3690c7fdf6cc63b) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:19:43.605462 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeab3cb-ac5e-4f96-b2f0-0f3a389916ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:43.607585 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeab3cb-ac5e-4f96-b2f0-0f3a389916ec None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:43.613781 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 73/139] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 15 07:19:43 2022] GET /identity/v3/role_assignments?user.id=7fdb3deb1979491fa3690c7fdf6cc63b&scope.project.id=659ee9038adf47e585b94881867c18c2&role.id=81b8a5256ba14184beb300947fbfe335 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:19:43.827597 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589881ec-fb5f-49a8-87d6-51e51b97f6db None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:43.948560 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589881ec-fb5f-49a8-87d6-51e51b97f6db None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:43.984822 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7a094e-ae5d-4feb-ada8-39c37188f975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:43.985479 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7a094e-ae5d-4feb-ada8-39c37188f975 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:43.985795 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7a094e-ae5d-4feb-ada8-39c37188f975 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:43.986075 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7a094e-ae5d-4feb-ada8-39c37188f975 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:43.987202 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7a094e-ae5d-4feb-ada8-39c37188f975 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:43.988660 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7a094e-ae5d-4feb-ada8-39c37188f975 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:43.990278 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 67/140] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:19:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:19:43.998673 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeab3cb-ac5e-4f96-b2f0-0f3a389916ec None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:43.999231 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bc70eb-39e1-4ff5-8e8e-2abe919a7ef9 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:43.999529 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bc70eb-39e1-4ff5-8e8e-2abe919a7ef9 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:43.999809 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bc70eb-39e1-4ff5-8e8e-2abe919a7ef9 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:44.001291 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 68/141] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:19:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:19:44.006693 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83df77b0-c96c-4140-883b-cc44d3ec5441 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:44.006981 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83df77b0-c96c-4140-883b-cc44d3ec5441 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:44.007265 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83df77b0-c96c-4140-883b-cc44d3ec5441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:44.028322 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-83df77b0-c96c-4140-883b-cc44d3ec5441 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:19:44.029698 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-83df77b0-c96c-4140-883b-cc44d3ec5441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:19:44.076241 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:44.103509 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 69/142] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:19:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 07:19:44.173089 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeab3cb-ac5e-4f96-b2f0-0f3a389916ec None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:44.227551 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdf1867-12c6-4b72-8667-7e81e21046a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:44.228171 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdf1867-12c6-4b72-8667-7e81e21046a5 None admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:44.228450 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdf1867-12c6-4b72-8667-7e81e21046a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:44.228727 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdf1867-12c6-4b72-8667-7e81e21046a5 None admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:44.229384 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdf1867-12c6-4b72-8667-7e81e21046a5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:44.230874 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdf1867-12c6-4b72-8667-7e81e21046a5 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:44.348351 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 74/143] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Sep 15 07:19:43 2022] POST /identity/v3/users => generated 283 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:19:44.556580 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83df77b0-c96c-4140-883b-cc44d3ec5441 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:44.699012 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83df77b0-c96c-4140-883b-cc44d3ec5441 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:44.753202 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2528d6-76a0-41f5-8645-7ce7cee7fdc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:44.754152 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2528d6-76a0-41f5-8645-7ce7cee7fdc0 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:44.754581 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2528d6-76a0-41f5-8645-7ce7cee7fdc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:44.755029 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2528d6-76a0-41f5-8645-7ce7cee7fdc0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:44.767908 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2528d6-76a0-41f5-8645-7ce7cee7fdc0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:19:44.777237 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-8a2528d6-76a0-41f5-8645-7ce7cee7fdc0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 07:19:44.777237 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:19:44.777237 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:19:44.777237 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:19:44.777237 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:19:44.777237 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:19:44.777237 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:19:44.777237 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:19:44.777237 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:19:44.777237 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:19:44.777237 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:19:44.777237 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:19:44.777237 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:19:44.777237 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:19:44.777237 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:19:44.777237 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:19:44.778287 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:19:44.778287 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:19:44.778287 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:19:44.778287 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:19:44.778287 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:19:44.778287 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:19:44.778287 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:19:44.778287 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:19:44.778287 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:19:44.778287 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:19:44.778287 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:19:44.778287 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:19:44.778287 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:19:44.778287 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:19:44.778287 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:19:44.778287 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:19:44.779293 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:19:44.779293 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:19:44.779293 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:19:44.779293 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:19:44.779293 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:19:44.779293 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:19:44.779293 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:19:44.779293 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:19:44.779293 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:19:44.779293 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:19:44.779293 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 07:19:44.779293 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:19:44.781408 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 70/144] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 15 07:19:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:19:44.787133 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdf1867-12c6-4b72-8667-7e81e21046a5 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:44.817645 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:44.871644 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181f8740-a99c-49f3-bb06-31218f950a88 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:44.872333 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181f8740-a99c-49f3-bb06-31218f950a88 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:44.872833 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181f8740-a99c-49f3-bb06-31218f950a88 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:44.874461 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 71/145] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:19:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 07:19:44.887006 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8127a00-6166-4f99-8f09-8e1a8c98d872 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:44.888352 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8127a00-6166-4f99-8f09-8e1a8c98d872 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:44.890830 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8127a00-6166-4f99-8f09-8e1a8c98d872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:44.907723 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdf1867-12c6-4b72-8667-7e81e21046a5 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:44.915718 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-d8127a00-6166-4f99-8f09-8e1a8c98d872 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:19:44.917259 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-d8127a00-6166-4f99-8f09-8e1a8c98d872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:19:44.945900 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff13dbc-6f26-46f2-a503-abf112ac8522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:44.946528 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff13dbc-6f26-46f2-a503-abf112ac8522 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:44.946807 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff13dbc-6f26-46f2-a503-abf112ac8522 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:44.947079 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff13dbc-6f26-46f2-a503-abf112ac8522 None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:44.947742 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff13dbc-6f26-46f2-a503-abf112ac8522 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:19:44.948040 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff13dbc-6f26-46f2-a503-abf112ac8522 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:44.949544 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff13dbc-6f26-46f2-a503-abf112ac8522 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:44.958809 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 75/146] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 15 07:19:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:19:44.965622 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:44.968523 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff13dbc-6f26-46f2-a503-abf112ac8522 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:45.026426 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 72/147] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:19:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 07:19:45.107182 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:45.131706 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff13dbc-6f26-46f2-a503-abf112ac8522 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:45.171739 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a526ca5-8cfa-4ad1-939e-d4e9da0fd50b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:45.172429 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a526ca5-8cfa-4ad1-939e-d4e9da0fd50b None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:45.172711 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a526ca5-8cfa-4ad1-939e-d4e9da0fd50b None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:45.172977 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a526ca5-8cfa-4ad1-939e-d4e9da0fd50b None admin] PATH_INFO: `/v3/users/06cea60dcdfb4c5ead6de8db2f0fd82f` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:45.175980 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a526ca5-8cfa-4ad1-939e-d4e9da0fd50b None admin] RBAC: Authorizing `identity:get_user(user_id=06cea60dcdfb4c5ead6de8db2f0fd82f)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:45.177501 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a526ca5-8cfa-4ad1-939e-d4e9da0fd50b None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:45.179268 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 76/148] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 15 07:19:44 2022] GET /identity/v3/users/06cea60dcdfb4c5ead6de8db2f0fd82f => generated 247 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:19:45.185787 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8127a00-6166-4f99-8f09-8e1a8c98d872 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:45.292246 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8127a00-6166-4f99-8f09-8e1a8c98d872 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:45.329206 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3627ff-22de-429d-a01e-fa3d27f0d8ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:45.329858 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3627ff-22de-429d-a01e-fa3d27f0d8ba None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:45.330145 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3627ff-22de-429d-a01e-fa3d27f0d8ba None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:45.330418 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3627ff-22de-429d-a01e-fa3d27f0d8ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:45.366629 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:45.401182 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3627ff-22de-429d-a01e-fa3d27f0d8ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:45.402710 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3627ff-22de-429d-a01e-fa3d27f0d8ba None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:45.506478 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 73/149] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 15 07:19:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 07:19:45.512061 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a526ca5-8cfa-4ad1-939e-d4e9da0fd50b None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:45.597949 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3627ff-22de-429d-a01e-fa3d27f0d8ba None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:45.626607 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a526ca5-8cfa-4ad1-939e-d4e9da0fd50b None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:45.703010 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eadd244-4703-463a-8d50-28a8784a5aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:45.704198 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eadd244-4703-463a-8d50-28a8784a5aa5 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:45.705917 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eadd244-4703-463a-8d50-28a8784a5aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:45.706415 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eadd244-4703-463a-8d50-28a8784a5aa5 None admin] PATH_INFO: `/v3/projects/659ee9038adf47e585b94881867c18c2` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:45.710259 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eadd244-4703-463a-8d50-28a8784a5aa5 None admin] RBAC: Authorizing `identity:get_project(project_id=659ee9038adf47e585b94881867c18c2)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:45.711937 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eadd244-4703-463a-8d50-28a8784a5aa5 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:45.716975 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 77/150] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 15 07:19:45 2022] GET /identity/v3/projects/659ee9038adf47e585b94881867c18c2 => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:19:45.723229 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eadd244-4703-463a-8d50-28a8784a5aa5 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:45.757953 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:45.772277 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:45.825059 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3627ff-22de-429d-a01e-fa3d27f0d8ba None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:45.893939 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87496482-542c-42ab-a615-9591232b8aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:45.895081 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87496482-542c-42ab-a615-9591232b8aef None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:45.896350 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87496482-542c-42ab-a615-9591232b8aef None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:45.896990 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87496482-542c-42ab-a615-9591232b8aef None admin] PATH_INFO: `/v3/roles/member` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:45.926459 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87496482-542c-42ab-a615-9591232b8aef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:19:45.935459 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-87496482-542c-42ab-a615-9591232b8aef None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 07:19:45.935459 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:19:45.935459 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:19:45.935459 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:19:45.935459 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:19:45.935459 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:19:45.935459 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:19:45.935459 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:19:45.935459 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:19:45.935459 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:19:45.935459 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:19:45.935459 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:19:45.935459 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:19:45.935459 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:19:45.935459 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:19:45.935459 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:19:45.936322 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:19:45.936322 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:19:45.936322 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:19:45.936322 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:19:45.936322 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:19:45.936322 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:19:45.936322 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:19:45.936322 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:19:45.936322 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:19:45.936322 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:19:45.936322 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:19:45.936322 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:19:45.936322 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:19:45.936322 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:19:45.936322 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:19:45.936322 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:19:45.937079 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:19:45.937079 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:19:45.937079 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:19:45.937079 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:19:45.937079 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:19:45.937079 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:19:45.937079 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:19:45.937079 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:19:45.937079 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:19:45.937079 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:19:45.937079 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 07:19:45.937079 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:19:45.942290 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eadd244-4703-463a-8d50-28a8784a5aa5 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:45.944772 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 74/151] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 15 07:19:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:19:45.957463 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87496482-542c-42ab-a615-9591232b8aef None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:46.033559 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b159884-7bbb-4f55-a41f-96ac06f8af95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:46.034212 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b159884-7bbb-4f55-a41f-96ac06f8af95 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:46.034492 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b159884-7bbb-4f55-a41f-96ac06f8af95 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:46.034766 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b159884-7bbb-4f55-a41f-96ac06f8af95 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:46.035507 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b159884-7bbb-4f55-a41f-96ac06f8af95 None admin] RBAC: Adding query filter params (role.id=6bbb643e0d8d4bc0a9b27c26edf59216, scope.project.id=659ee9038adf47e585b94881867c18c2, user.id=06cea60dcdfb4c5ead6de8db2f0fd82f) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:19:46.035817 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b159884-7bbb-4f55-a41f-96ac06f8af95 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:46.037269 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b159884-7bbb-4f55-a41f-96ac06f8af95 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:46.057921 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 78/152] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 15 07:19:45 2022] GET /identity/v3/role_assignments?user.id=06cea60dcdfb4c5ead6de8db2f0fd82f&scope.project.id=659ee9038adf47e585b94881867c18c2&role.id=6bbb643e0d8d4bc0a9b27c26edf59216 => generated 608 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:19:46.128290 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87496482-542c-42ab-a615-9591232b8aef None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:46.167445 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:46.168942 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb04e0f7-10ef-4076-a6b1-d6b7c3068c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:46.170195 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb04e0f7-10ef-4076-a6b1-d6b7c3068c8a None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:46.170479 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb04e0f7-10ef-4076-a6b1-d6b7c3068c8a None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:46.170809 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb04e0f7-10ef-4076-a6b1-d6b7c3068c8a None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:46.171445 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb04e0f7-10ef-4076-a6b1-d6b7c3068c8a None admin] RBAC: Adding query filter params (name=member) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:19:46.171759 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb04e0f7-10ef-4076-a6b1-d6b7c3068c8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:46.173170 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb04e0f7-10ef-4076-a6b1-d6b7c3068c8a None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:46.186102 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 75/153] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Sep 15 07:19:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:19:46.192357 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b159884-7bbb-4f55-a41f-96ac06f8af95 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:46.250477 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca0b13-86f7-4e7c-aed4-7cb3fc31140b None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:46.251161 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca0b13-86f7-4e7c-aed4-7cb3fc31140b None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:46.251644 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca0b13-86f7-4e7c-aed4-7cb3fc31140b None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:46.253259 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 76/154] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:19:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:19:46.267062 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565e1fe9-ec6a-44a0-9591-768a404cc52c None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:46.267671 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565e1fe9-ec6a-44a0-9591-768a404cc52c None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:46.268152 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565e1fe9-ec6a-44a0-9591-768a404cc52c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:46.292064 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:46.299102 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-565e1fe9-ec6a-44a0-9591-768a404cc52c None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:19:46.300698 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-565e1fe9-ec6a-44a0-9591-768a404cc52c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:19:46.320423 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b159884-7bbb-4f55-a41f-96ac06f8af95 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:46.386937 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d696fb-e086-46b8-acee-6ca7aa6d66d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:46.388412 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d696fb-e086-46b8-acee-6ca7aa6d66d6 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:46.388839 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d696fb-e086-46b8-acee-6ca7aa6d66d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:46.389349 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d696fb-e086-46b8-acee-6ca7aa6d66d6 None admin] PATH_INFO: `/v3/users/001806a30226487ab9e17f8d988f1ca6` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:46.391890 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d696fb-e086-46b8-acee-6ca7aa6d66d6 None admin] RBAC: Authorizing `identity:get_user(user_id=001806a30226487ab9e17f8d988f1ca6)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:46.393785 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d696fb-e086-46b8-acee-6ca7aa6d66d6 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:46.395885 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 79/155] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 15 07:19:46 2022] GET /identity/v3/users/001806a30226487ab9e17f8d988f1ca6 => generated 275 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:19:46.401731 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d696fb-e086-46b8-acee-6ca7aa6d66d6 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:46.427223 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 77/156] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:19:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 07:19:46.550585 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d696fb-e086-46b8-acee-6ca7aa6d66d6 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:46.589076 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9efabd9-bc54-495c-9d2d-d17d248f06b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:46.589968 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9efabd9-bc54-495c-9d2d-d17d248f06b0 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:46.590386 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9efabd9-bc54-495c-9d2d-d17d248f06b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:46.590794 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9efabd9-bc54-495c-9d2d-d17d248f06b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:46.613553 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:46.623995 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed5ea3-604c-409e-ba00-be2c448b2955 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:46.624884 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed5ea3-604c-409e-ba00-be2c448b2955 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:46.625329 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed5ea3-604c-409e-ba00-be2c448b2955 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:46.626900 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 78/157] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:19:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:19:46.632432 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ac3c55-0940-4b14-9c36-b4b6e8338339 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:46.641710 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ac3c55-0940-4b14-9c36-b4b6e8338339 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:46.641710 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ac3c55-0940-4b14-9c36-b4b6e8338339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:46.668576 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9efabd9-bc54-495c-9d2d-d17d248f06b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:46.670091 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-60ac3c55-0940-4b14-9c36-b4b6e8338339 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:19:46.670221 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9efabd9-bc54-495c-9d2d-d17d248f06b0 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:46.671639 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-60ac3c55-0940-4b14-9c36-b4b6e8338339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:19:46.791564 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 80/158] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 15 07:19:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 15 07:19:46.797410 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9efabd9-bc54-495c-9d2d-d17d248f06b0 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:46.875346 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 79/159] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:19:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 07:19:46.891633 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:46.957391 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9efabd9-bc54-495c-9d2d-d17d248f06b0 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:46.974573 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ac3c55-0940-4b14-9c36-b4b6e8338339 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:47.027459 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa189c5-d079-43b2-adee-f428525475d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:47.028464 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa189c5-d079-43b2-adee-f428525475d0 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:47.029707 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa189c5-d079-43b2-adee-f428525475d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:47.030191 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa189c5-d079-43b2-adee-f428525475d0 None admin] PATH_INFO: `/v3/projects/659ee9038adf47e585b94881867c18c2` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:47.034278 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa189c5-d079-43b2-adee-f428525475d0 None admin] RBAC: Authorizing `identity:get_project(project_id=659ee9038adf47e585b94881867c18c2)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:47.035979 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa189c5-d079-43b2-adee-f428525475d0 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:47.038169 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 81/160] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 15 07:19:46 2022] GET /identity/v3/projects/659ee9038adf47e585b94881867c18c2 => generated 298 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:19:47.039535 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77df535c-c8ec-452e-8f6e-a296bd61eb9e None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:47.040017 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77df535c-c8ec-452e-8f6e-a296bd61eb9e None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:47.040473 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77df535c-c8ec-452e-8f6e-a296bd61eb9e None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:47.042002 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 82/161] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:19:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:19:47.050522 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77df535c-c8ec-452e-8f6e-a296bd61eb9e None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:47.065900 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:47.123630 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ac3c55-0940-4b14-9c36-b4b6e8338339 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:47.162281 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca75f12-1a77-4f17-8742-be07f227951e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:47.163419 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca75f12-1a77-4f17-8742-be07f227951e None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:47.164680 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca75f12-1a77-4f17-8742-be07f227951e None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:47.165121 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca75f12-1a77-4f17-8742-be07f227951e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:47.177817 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca75f12-1a77-4f17-8742-be07f227951e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:19:47.183803 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-1ca75f12-1a77-4f17-8742-be07f227951e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 07:19:47.183803 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:19:47.183803 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:19:47.183803 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:19:47.183803 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:19:47.183803 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:19:47.183803 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:19:47.183803 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:19:47.183803 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:19:47.183803 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:19:47.183803 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:19:47.183803 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:19:47.183803 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:19:47.183803 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:19:47.183803 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:19:47.183803 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:19:47.185040 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:19:47.185040 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:19:47.185040 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:19:47.185040 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:19:47.185040 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:19:47.185040 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:19:47.185040 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:19:47.185040 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:19:47.185040 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:19:47.185040 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:19:47.185040 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:19:47.185040 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:19:47.185040 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:19:47.185040 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:19:47.185040 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:19:47.185040 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:19:47.189740 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:19:47.189740 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:19:47.189740 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:19:47.189740 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:19:47.189740 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:19:47.189740 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:19:47.189740 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:19:47.189740 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:19:47.189740 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:19:47.189740 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:19:47.189740 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 07:19:47.189740 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:19:47.189740 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 80/162] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 15 07:19:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 07:19:47.189740 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3178b5c-632d-4470-91dc-47451dfae788 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:47.190640 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3178b5c-632d-4470-91dc-47451dfae788 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:47.190916 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3178b5c-632d-4470-91dc-47451dfae788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:47.211436 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-c3178b5c-632d-4470-91dc-47451dfae788 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:19:47.212904 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-c3178b5c-632d-4470-91dc-47451dfae788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:19:47.325662 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77df535c-c8ec-452e-8f6e-a296bd61eb9e None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:47.351291 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 81/163] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:19:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 07:19:47.352158 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3178b5c-632d-4470-91dc-47451dfae788 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:47.364118 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:47.418298 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068c0428-7a50-4f40-a44f-51de48e1f5cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:47.418922 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068c0428-7a50-4f40-a44f-51de48e1f5cc None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:47.419828 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068c0428-7a50-4f40-a44f-51de48e1f5cc None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:47.420104 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068c0428-7a50-4f40-a44f-51de48e1f5cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:47.420987 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068c0428-7a50-4f40-a44f-51de48e1f5cc None admin] RBAC: Adding query filter params (role.id=053e12af84e943e699460a9d76c2072b, scope.project.id=659ee9038adf47e585b94881867c18c2, user.id=001806a30226487ab9e17f8d988f1ca6) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:19:47.421293 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068c0428-7a50-4f40-a44f-51de48e1f5cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:47.422790 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068c0428-7a50-4f40-a44f-51de48e1f5cc None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:47.436878 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 83/164] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 15 07:19:47 2022] GET /identity/v3/role_assignments?user.id=001806a30226487ab9e17f8d988f1ca6&scope.project.id=659ee9038adf47e585b94881867c18c2&role.id=053e12af84e943e699460a9d76c2072b => generated 608 bytes in 387 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:19:47.437510 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068c0428-7a50-4f40-a44f-51de48e1f5cc None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:47.446701 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:47.576208 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3178b5c-632d-4470-91dc-47451dfae788 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:47.649940 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4862baee-f82c-4d72-81f6-793d70d87cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:47.650595 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4862baee-f82c-4d72-81f6-793d70d87cb0 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:47.650856 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4862baee-f82c-4d72-81f6-793d70d87cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:47.651132 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4862baee-f82c-4d72-81f6-793d70d87cb0 None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:47.651827 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4862baee-f82c-4d72-81f6-793d70d87cb0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:19:47.652134 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4862baee-f82c-4d72-81f6-793d70d87cb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:47.654163 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4862baee-f82c-4d72-81f6-793d70d87cb0 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:47.670650 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 82/165] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Sep 15 07:19:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:19:47.674697 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4862baee-f82c-4d72-81f6-793d70d87cb0 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:47.736488 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068c0428-7a50-4f40-a44f-51de48e1f5cc None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:47.806295 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f4794f-9e9a-4cc5-8791-463767793689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:47.806950 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f4794f-9e9a-4cc5-8791-463767793689 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:47.807233 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f4794f-9e9a-4cc5-8791-463767793689 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:47.807508 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f4794f-9e9a-4cc5-8791-463767793689 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:47.826821 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4862baee-f82c-4d72-81f6-793d70d87cb0 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:47.828775 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:47.830561 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f4794f-9e9a-4cc5-8791-463767793689 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:19:47.835989 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-63f4794f-9e9a-4cc5-8791-463767793689 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 07:19:47.835989 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:19:47.835989 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:19:47.835989 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:19:47.835989 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:19:47.835989 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:19:47.835989 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:19:47.835989 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:19:47.835989 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:19:47.835989 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:19:47.835989 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:19:47.835989 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:19:47.835989 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:19:47.835989 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:19:47.835989 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:19:47.835989 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:19:47.846500 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:19:47.846500 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:19:47.846500 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:19:47.846500 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:19:47.846500 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:19:47.846500 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:19:47.846500 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:19:47.846500 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:19:47.846500 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:19:47.846500 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:19:47.846500 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:19:47.846500 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:19:47.846500 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:19:47.846500 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:19:47.846500 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:19:47.846500 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:19:47.847443 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:19:47.847443 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:19:47.847443 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:19:47.847443 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:19:47.847443 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:19:47.847443 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:19:47.847443 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:19:47.847443 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:19:47.847443 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:19:47.847443 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:19:47.847443 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 07:19:47.847443 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:19:47.847443 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 84/166] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Sep 15 07:19:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 400 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:19:47.847443 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f4794f-9e9a-4cc5-8791-463767793689 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:47.867378 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27b9d16-2af1-4816-9801-da42f307eba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:47.868331 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27b9d16-2af1-4816-9801-da42f307eba9 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:47.868758 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27b9d16-2af1-4816-9801-da42f307eba9 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:47.869176 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27b9d16-2af1-4816-9801-da42f307eba9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:47.880934 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27b9d16-2af1-4816-9801-da42f307eba9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:19:47.886221 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-e27b9d16-2af1-4816-9801-da42f307eba9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 07:19:47.886221 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:19:47.886221 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:19:47.886221 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:19:47.886221 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:19:47.886221 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:19:47.886221 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:19:47.886221 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:19:47.886221 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:19:47.886221 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:19:47.886221 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:19:47.886221 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:19:47.886221 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:19:47.886221 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:19:47.886221 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:19:47.886221 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:19:47.887309 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:19:47.887309 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:19:47.887309 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:19:47.887309 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:19:47.887309 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:19:47.887309 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:19:47.887309 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:19:47.887309 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:19:47.887309 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:19:47.887309 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:19:47.887309 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:19:47.887309 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:19:47.887309 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:19:47.887309 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:19:47.887309 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:19:47.887309 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:19:47.888735 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:19:47.888735 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:19:47.888735 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:19:47.888735 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:19:47.888735 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:19:47.888735 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:19:47.888735 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:19:47.888735 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:19:47.888735 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:19:47.888735 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:19:47.888735 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 07:19:47.888735 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:19:47.891085 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 83/167] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 15 07:19:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:19:47.892137 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27b9d16-2af1-4816-9801-da42f307eba9 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:47.925128 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:48.006902 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27b9d16-2af1-4816-9801-da42f307eba9 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:48.037779 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f4794f-9e9a-4cc5-8791-463767793689 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:48.045871 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346c5851-88c4-4910-b874-bd823d2d12b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:48.046697 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346c5851-88c4-4910-b874-bd823d2d12b2 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:48.047133 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346c5851-88c4-4910-b874-bd823d2d12b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:48.047562 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346c5851-88c4-4910-b874-bd823d2d12b2 None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:48.048397 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346c5851-88c4-4910-b874-bd823d2d12b2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:19:48.048852 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346c5851-88c4-4910-b874-bd823d2d12b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:48.050443 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346c5851-88c4-4910-b874-bd823d2d12b2 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:48.060629 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 84/168] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 15 07:19:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:19:48.061913 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346c5851-88c4-4910-b874-bd823d2d12b2 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:48.078697 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62a15d2-c8c3-452f-9789-6788190723c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:48.079504 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62a15d2-c8c3-452f-9789-6788190723c3 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:48.079628 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62a15d2-c8c3-452f-9789-6788190723c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:48.079827 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62a15d2-c8c3-452f-9789-6788190723c3 None admin] PATH_INFO: `/v3/users/001806a30226487ab9e17f8d988f1ca6` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:48.081026 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62a15d2-c8c3-452f-9789-6788190723c3 None admin] RBAC: Authorizing `identity:get_user(user_id=001806a30226487ab9e17f8d988f1ca6)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:48.082521 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62a15d2-c8c3-452f-9789-6788190723c3 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:48.084018 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 85/169] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 15 07:19:47 2022] GET /identity/v3/users/001806a30226487ab9e17f8d988f1ca6 => generated 275 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:19:48.084782 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62a15d2-c8c3-452f-9789-6788190723c3 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:48.092037 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:48.173445 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346c5851-88c4-4910-b874-bd823d2d12b2 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:48.195592 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62a15d2-c8c3-452f-9789-6788190723c3 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:48.206632 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:48.211564 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a3447a-85b9-4fb1-9385-df368051ec51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:48.212367 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a3447a-85b9-4fb1-9385-df368051ec51 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:48.212811 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a3447a-85b9-4fb1-9385-df368051ec51 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:48.213233 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a3447a-85b9-4fb1-9385-df368051ec51 None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:48.214147 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a3447a-85b9-4fb1-9385-df368051ec51 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:19:48.214666 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a3447a-85b9-4fb1-9385-df368051ec51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:48.217311 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a3447a-85b9-4fb1-9385-df368051ec51 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:48.226512 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 85/170] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Sep 15 07:19:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:19:48.227752 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a3447a-85b9-4fb1-9385-df368051ec51 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:48.233336 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5127b7-8d92-4852-a8e9-f23ec1847fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:48.233720 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5127b7-8d92-4852-a8e9-f23ec1847fd0 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:48.234016 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5127b7-8d92-4852-a8e9-f23ec1847fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:48.234298 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5127b7-8d92-4852-a8e9-f23ec1847fd0 None admin] PATH_INFO: `/v3/users/001806a30226487ab9e17f8d988f1ca6` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:48.235631 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5127b7-8d92-4852-a8e9-f23ec1847fd0 None admin] RBAC: Authorizing `identity:get_user(user_id=001806a30226487ab9e17f8d988f1ca6)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:48.237112 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5127b7-8d92-4852-a8e9-f23ec1847fd0 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:48.239045 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 86/171] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 15 07:19:48 2022] GET /identity/v3/users/001806a30226487ab9e17f8d988f1ca6 => generated 275 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:19:48.240213 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5127b7-8d92-4852-a8e9-f23ec1847fd0 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:48.321475 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:48.337991 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a3447a-85b9-4fb1-9385-df368051ec51 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:48.346811 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5127b7-8d92-4852-a8e9-f23ec1847fd0 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:48.375406 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db567438-02de-48bd-bb03-3a4d24f967a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:48.376252 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db567438-02de-48bd-bb03-3a4d24f967a0 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:48.376729 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db567438-02de-48bd-bb03-3a4d24f967a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:48.377155 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db567438-02de-48bd-bb03-3a4d24f967a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:48.383388 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533df145-3f19-42de-a170-ec8a320ef2f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:48.384149 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533df145-3f19-42de-a170-ec8a320ef2f6 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:48.384534 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533df145-3f19-42de-a170-ec8a320ef2f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:48.384911 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533df145-3f19-42de-a170-ec8a320ef2f6 None admin] PATH_INFO: `/v3/users/7fdb3deb1979491fa3690c7fdf6cc63b` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:48.386179 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533df145-3f19-42de-a170-ec8a320ef2f6 None admin] RBAC: Authorizing `identity:get_user(user_id=7fdb3deb1979491fa3690c7fdf6cc63b)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:48.387764 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533df145-3f19-42de-a170-ec8a320ef2f6 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:48.390237 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 87/172] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 15 07:19:48 2022] GET /identity/v3/users/7fdb3deb1979491fa3690c7fdf6cc63b => generated 289 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:19:48.390976 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533df145-3f19-42de-a170-ec8a320ef2f6 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:48.439940 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:48.452451 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db567438-02de-48bd-bb03-3a4d24f967a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:48.454004 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db567438-02de-48bd-bb03-3a4d24f967a0 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:48.498351 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533df145-3f19-42de-a170-ec8a320ef2f6 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:48.534307 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04727b9-4196-4b84-8f5a-c833fd4981a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:48.535303 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04727b9-4196-4b84-8f5a-c833fd4981a6 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:48.535824 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04727b9-4196-4b84-8f5a-c833fd4981a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:48.536310 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04727b9-4196-4b84-8f5a-c833fd4981a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:48.542565 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:48.560462 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 86/173] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 15 07:19:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 07:19:48.561566 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db567438-02de-48bd-bb03-3a4d24f967a0 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:48.609943 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04727b9-4196-4b84-8f5a-c833fd4981a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:48.611741 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04727b9-4196-4b84-8f5a-c833fd4981a6 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:48.659129 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:48.666589 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db567438-02de-48bd-bb03-3a4d24f967a0 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:48.702756 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4044b70-1b71-4019-ae5f-c3d2bcf79183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:48.703399 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4044b70-1b71-4019-ae5f-c3d2bcf79183 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:48.703678 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4044b70-1b71-4019-ae5f-c3d2bcf79183 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:48.703948 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4044b70-1b71-4019-ae5f-c3d2bcf79183 None admin] PATH_INFO: `/v3/projects/659ee9038adf47e585b94881867c18c2` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:48.705397 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4044b70-1b71-4019-ae5f-c3d2bcf79183 None admin] RBAC: Authorizing `identity:get_project(project_id=659ee9038adf47e585b94881867c18c2)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:48.706630 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4044b70-1b71-4019-ae5f-c3d2bcf79183 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:48.708517 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 87/174] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 15 07:19:48 2022] GET /identity/v3/projects/659ee9038adf47e585b94881867c18c2 => generated 298 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:19:48.709104 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4044b70-1b71-4019-ae5f-c3d2bcf79183 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:48.720314 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 88/175] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 15 07:19:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 07:19:48.721443 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04727b9-4196-4b84-8f5a-c833fd4981a6 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:48.772659 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:48.831286 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4044b70-1b71-4019-ae5f-c3d2bcf79183 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:48.848741 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04727b9-4196-4b84-8f5a-c833fd4981a6 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:48.880250 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a4aab0-d9e3-4092-87db-a5d254fa5f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:48.881221 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a4aab0-d9e3-4092-87db-a5d254fa5f1e None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:48.881828 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a4aab0-d9e3-4092-87db-a5d254fa5f1e None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:48.882345 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a4aab0-d9e3-4092-87db-a5d254fa5f1e None admin] PATH_INFO: `/v3/projects/adc861e34dc44f389899da5a1c6f0770` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:48.883918 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a4aab0-d9e3-4092-87db-a5d254fa5f1e None admin] RBAC: Authorizing `identity:get_project(project_id=adc861e34dc44f389899da5a1c6f0770)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:48.884874 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661b5664-a3cc-4013-886e-7d85359362fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:48.885543 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661b5664-a3cc-4013-886e-7d85359362fa None admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:48.885859 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661b5664-a3cc-4013-886e-7d85359362fa None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:48.886135 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661b5664-a3cc-4013-886e-7d85359362fa None admin] PATH_INFO: `/v3/projects/659ee9038adf47e585b94881867c18c2/users/7fdb3deb1979491fa3690c7fdf6cc63b/roles/81b8a5256ba14184beb300947fbfe335` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:48.886532 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a4aab0-d9e3-4092-87db-a5d254fa5f1e None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:48.888111 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661b5664-a3cc-4013-886e-7d85359362fa None admin] RBAC: Authorizing `identity:create_grant(project_id=659ee9038adf47e585b94881867c18c2, user_id=7fdb3deb1979491fa3690c7fdf6cc63b, role_id=81b8a5256ba14184beb300947fbfe335)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:48.888704 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 88/176] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 15 07:19:48 2022] GET /identity/v3/projects/adc861e34dc44f389899da5a1c6f0770 => generated 312 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:19:48.889565 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661b5664-a3cc-4013-886e-7d85359362fa None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:48.890063 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a4aab0-d9e3-4092-87db-a5d254fa5f1e None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:48.960867 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 89/177] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Sep 15 07:19:48 2022] PUT /identity/v3/projects/659ee9038adf47e585b94881867c18c2/users/7fdb3deb1979491fa3690c7fdf6cc63b/roles/81b8a5256ba14184beb300947fbfe335 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:19:48.964729 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bac041-e409-47f1-9331-cb6b80a055fd None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:48.965081 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bac041-e409-47f1-9331-cb6b80a055fd None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:48.965470 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bac041-e409-47f1-9331-cb6b80a055fd None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:48.968049 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 90/178] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:19:48 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:19:48.968940 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bac041-e409-47f1-9331-cb6b80a055fd None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:48.992107 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:49.003493 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a4aab0-d9e3-4092-87db-a5d254fa5f1e None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:49.029228 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:49.041976 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6eebc3-7816-46cb-9f3f-9d8a524a4137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:49.043000 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6eebc3-7816-46cb-9f3f-9d8a524a4137 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:49.043522 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6eebc3-7816-46cb-9f3f-9d8a524a4137 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:49.043939 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6eebc3-7816-46cb-9f3f-9d8a524a4137 None admin] PATH_INFO: `/v3/projects/659ee9038adf47e585b94881867c18c2` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:49.045898 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6eebc3-7816-46cb-9f3f-9d8a524a4137 None admin] RBAC: Authorizing `identity:get_project(project_id=659ee9038adf47e585b94881867c18c2)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:49.047511 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6eebc3-7816-46cb-9f3f-9d8a524a4137 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:49.049376 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 89/179] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 15 07:19:48 2022] GET /identity/v3/projects/659ee9038adf47e585b94881867c18c2 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:19:49.050856 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441770b9-ca9e-438e-a20c-30a60f9b433a None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:49.051337 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441770b9-ca9e-438e-a20c-30a60f9b433a None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:49.051795 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441770b9-ca9e-438e-a20c-30a60f9b433a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:49.072434 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-441770b9-ca9e-438e-a20c-30a60f9b433a None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:19:49.074111 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-441770b9-ca9e-438e-a20c-30a60f9b433a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:19:49.154479 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 90/180] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:19:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 07:19:49.155791 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441770b9-ca9e-438e-a20c-30a60f9b433a None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:49.167363 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bac041-e409-47f1-9331-cb6b80a055fd None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:49.216055 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e5ed15-fb63-40e1-9b45-718883ef5757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:49.218158 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e5ed15-fb63-40e1-9b45-718883ef5757 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:49.218441 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e5ed15-fb63-40e1-9b45-718883ef5757 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:49.218711 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e5ed15-fb63-40e1-9b45-718883ef5757 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:49.219498 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e5ed15-fb63-40e1-9b45-718883ef5757 None admin] RBAC: Adding query filter params (role.id=053e12af84e943e699460a9d76c2072b, scope.project.id=adc861e34dc44f389899da5a1c6f0770, user.id=001806a30226487ab9e17f8d988f1ca6) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:19:49.219813 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e5ed15-fb63-40e1-9b45-718883ef5757 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:49.221313 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e5ed15-fb63-40e1-9b45-718883ef5757 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:49.238596 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 91/181] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 15 07:19:48 2022] GET /identity/v3/role_assignments?user.id=001806a30226487ab9e17f8d988f1ca6&scope.project.id=adc861e34dc44f389899da5a1c6f0770&role.id=053e12af84e943e699460a9d76c2072b => generated 608 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:19:49.336145 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441770b9-ca9e-438e-a20c-30a60f9b433a None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:49.366369 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:49.387378 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1643a7-a3d0-40a4-ad37-8183cd864dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:49.388737 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1643a7-a3d0-40a4-ad37-8183cd864dde None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:49.389220 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1643a7-a3d0-40a4-ad37-8183cd864dde None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:49.389767 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1643a7-a3d0-40a4-ad37-8183cd864dde None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:49.390922 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1643a7-a3d0-40a4-ad37-8183cd864dde None admin] RBAC: Adding query filter params (role.id=a0fae926490542b4977d1ecf27fcece4, scope.project.id=659ee9038adf47e585b94881867c18c2, user.id=001806a30226487ab9e17f8d988f1ca6) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:19:49.391238 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1643a7-a3d0-40a4-ad37-8183cd864dde None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:49.396550 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1643a7-a3d0-40a4-ad37-8183cd864dde None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:49.406575 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 91/182] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 15 07:19:49 2022] GET /identity/v3/role_assignments?user.id=001806a30226487ab9e17f8d988f1ca6&scope.project.id=659ee9038adf47e585b94881867c18c2&role.id=a0fae926490542b4977d1ecf27fcece4 => generated 608 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:19:49.575179 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e5ed15-fb63-40e1-9b45-718883ef5757 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:49.662471 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:49.703931 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e5ed15-fb63-40e1-9b45-718883ef5757 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:49.744408 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2954832-3104-4db0-b147-8fb357535dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:49.745054 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2954832-3104-4db0-b147-8fb357535dde None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:49.745335 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2954832-3104-4db0-b147-8fb357535dde None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:49.745645 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2954832-3104-4db0-b147-8fb357535dde None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:49.757141 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2954832-3104-4db0-b147-8fb357535dde None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:19:49.763388 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-d2954832-3104-4db0-b147-8fb357535dde None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 07:19:49.763388 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:19:49.763388 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:19:49.763388 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:19:49.763388 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:19:49.763388 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:19:49.763388 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:19:49.763388 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:19:49.763388 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:19:49.763388 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:19:49.763388 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:19:49.763388 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:19:49.763388 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:19:49.763388 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:19:49.763388 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:19:49.763388 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:19:49.764441 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:19:49.764441 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:19:49.764441 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:19:49.764441 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:19:49.764441 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:19:49.764441 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:19:49.764441 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:19:49.764441 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:19:49.764441 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:19:49.764441 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:19:49.764441 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:19:49.764441 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:19:49.764441 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:19:49.764441 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:19:49.764441 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:19:49.764441 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:19:49.765869 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:19:49.765869 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:19:49.765869 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:19:49.765869 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:19:49.765869 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:19:49.765869 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:19:49.765869 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:19:49.765869 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:19:49.765869 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:19:49.765869 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:19:49.765869 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 07:19:49.765869 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:19:49.765869 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 92/183] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 15 07:19:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:19:49.770329 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1643a7-a3d0-40a4-ad37-8183cd864dde None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:49.845983 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09715686-cf81-4cac-ab86-c5027743064b None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:49.846346 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09715686-cf81-4cac-ab86-c5027743064b None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:49.846529 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09715686-cf81-4cac-ab86-c5027743064b None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:49.847877 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 93/184] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:19:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 07:19:49.854009 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64db3d2a-9bd8-4f48-93b2-a0355e7c6150 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:49.854568 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64db3d2a-9bd8-4f48-93b2-a0355e7c6150 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:49.854568 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64db3d2a-9bd8-4f48-93b2-a0355e7c6150 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:49.855992 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 94/185] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:19:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 07:19:49.860477 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb915ef9-500f-45e0-b3c3-f336dbdf6ff7 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:49.860721 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb915ef9-500f-45e0-b3c3-f336dbdf6ff7 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:49.860988 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb915ef9-500f-45e0-b3c3-f336dbdf6ff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:49.884140 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-bb915ef9-500f-45e0-b3c3-f336dbdf6ff7 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:19:49.885336 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-bb915ef9-500f-45e0-b3c3-f336dbdf6ff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:19:49.891724 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1643a7-a3d0-40a4-ad37-8183cd864dde None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:49.928008 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3ae9be-ab49-49e6-b012-495a4411d99a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:49.929424 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3ae9be-ab49-49e6-b012-495a4411d99a None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:49.929811 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3ae9be-ab49-49e6-b012-495a4411d99a None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:49.930089 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3ae9be-ab49-49e6-b012-495a4411d99a None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:49.931023 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3ae9be-ab49-49e6-b012-495a4411d99a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:19:49.931328 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3ae9be-ab49-49e6-b012-495a4411d99a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:49.932699 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3ae9be-ab49-49e6-b012-495a4411d99a None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:49.941779 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 92/186] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 15 07:19:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:19:49.943017 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82db9f5c-7869-49a1-8b1c-8c9e48d43b95 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:49.943813 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82db9f5c-7869-49a1-8b1c-8c9e48d43b95 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:49.943813 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82db9f5c-7869-49a1-8b1c-8c9e48d43b95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:49.970028 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-82db9f5c-7869-49a1-8b1c-8c9e48d43b95 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:19:49.971748 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-82db9f5c-7869-49a1-8b1c-8c9e48d43b95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:19:49.989724 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 95/187] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:19:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 07:19:49.991275 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb915ef9-500f-45e0-b3c3-f336dbdf6ff7 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:50.018514 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:50.044047 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 93/188] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:19:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 07:19:50.070836 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:50.113998 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb915ef9-500f-45e0-b3c3-f336dbdf6ff7 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:50.150140 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5d6144-f57d-4191-96df-30125909662d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:50.151139 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5d6144-f57d-4191-96df-30125909662d None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:50.151622 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5d6144-f57d-4191-96df-30125909662d None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:50.152110 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5d6144-f57d-4191-96df-30125909662d None admin] PATH_INFO: `/v3/users/053b642391194fa899b9eca12ae3cf99` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:50.166497 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5d6144-f57d-4191-96df-30125909662d None admin] RBAC: Authorizing `identity:get_user(user_id=053b642391194fa899b9eca12ae3cf99)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:50.168297 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5d6144-f57d-4191-96df-30125909662d None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:50.170784 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 96/189] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 15 07:19:49 2022] GET /identity/v3/users/053b642391194fa899b9eca12ae3cf99 => generated 283 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:19:50.181074 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82db9f5c-7869-49a1-8b1c-8c9e48d43b95 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:50.283994 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5d6144-f57d-4191-96df-30125909662d None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:50.302865 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82db9f5c-7869-49a1-8b1c-8c9e48d43b95 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:50.346873 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1717cb9-e54e-4193-851d-8d67871fed65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:50.347486 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1717cb9-e54e-4193-851d-8d67871fed65 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:50.347764 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1717cb9-e54e-4193-851d-8d67871fed65 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:50.348032 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1717cb9-e54e-4193-851d-8d67871fed65 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:50.390870 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5d6144-f57d-4191-96df-30125909662d None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:50.398426 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:50.408135 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:50.422120 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1717cb9-e54e-4193-851d-8d67871fed65 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:50.423581 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1717cb9-e54e-4193-851d-8d67871fed65 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:50.424406 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71303450-d606-4bf1-a064-0e70b718b487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:50.425042 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71303450-d606-4bf1-a064-0e70b718b487 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:50.425324 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71303450-d606-4bf1-a064-0e70b718b487 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:50.425609 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71303450-d606-4bf1-a064-0e70b718b487 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:50.426733 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71303450-d606-4bf1-a064-0e70b718b487 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:50.428224 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71303450-d606-4bf1-a064-0e70b718b487 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:50.433703 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 97/190] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:19:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:19:50.433703 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71303450-d606-4bf1-a064-0e70b718b487 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:50.535047 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71303450-d606-4bf1-a064-0e70b718b487 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:50.540731 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 94/191] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 15 07:19:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 07:19:50.541414 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1717cb9-e54e-4193-851d-8d67871fed65 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:50.578556 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194a9c31-9813-4114-b031-f0f694f4fd8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:50.579187 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194a9c31-9813-4114-b031-f0f694f4fd8e None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:50.579463 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194a9c31-9813-4114-b031-f0f694f4fd8e None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:50.579733 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194a9c31-9813-4114-b031-f0f694f4fd8e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:50.589918 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194a9c31-9813-4114-b031-f0f694f4fd8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:19:50.595812 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-194a9c31-9813-4114-b031-f0f694f4fd8e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 07:19:50.595812 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:19:50.595812 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:19:50.595812 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:19:50.595812 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:19:50.595812 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:19:50.595812 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:19:50.595812 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:19:50.595812 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:19:50.595812 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:19:50.595812 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:19:50.595812 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:19:50.595812 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:19:50.595812 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:19:50.595812 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:19:50.595812 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:19:50.596711 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:19:50.596711 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:19:50.596711 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:19:50.596711 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:19:50.596711 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:19:50.596711 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:19:50.596711 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:19:50.596711 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:19:50.596711 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:19:50.596711 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:19:50.596711 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:19:50.596711 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:19:50.596711 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:19:50.596711 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:19:50.596711 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:19:50.596711 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:19:50.597495 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:19:50.597495 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:19:50.597495 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:19:50.597495 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:19:50.597495 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:19:50.597495 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:19:50.597495 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:19:50.597495 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:19:50.597495 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:19:50.597495 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:19:50.597495 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 07:19:50.597495 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:19:50.597495 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 98/192] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Sep 15 07:19:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:19:50.598782 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194a9c31-9813-4114-b031-f0f694f4fd8e None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:50.629499 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:50.651441 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:50.653309 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1717cb9-e54e-4193-851d-8d67871fed65 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:50.690564 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb5999f-95cf-4eef-b972-ee67face029f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:50.691088 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb5999f-95cf-4eef-b972-ee67face029f None admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:50.691367 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb5999f-95cf-4eef-b972-ee67face029f None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:50.691639 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb5999f-95cf-4eef-b972-ee67face029f None admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:50.692295 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb5999f-95cf-4eef-b972-ee67face029f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:50.693750 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb5999f-95cf-4eef-b972-ee67face029f None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:50.708251 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194a9c31-9813-4114-b031-f0f694f4fd8e None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:50.741199 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46743a32-8d0a-4de5-a1a5-a138381b9baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:50.741848 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46743a32-8d0a-4de5-a1a5-a138381b9baa None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:50.742130 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46743a32-8d0a-4de5-a1a5-a138381b9baa None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:50.742406 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46743a32-8d0a-4de5-a1a5-a138381b9baa None admin] PATH_INFO: `/v3/projects/91757d436677474ab5f1027263238244` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:50.743520 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46743a32-8d0a-4de5-a1a5-a138381b9baa None admin] RBAC: Authorizing `identity:get_project(project_id=91757d436677474ab5f1027263238244)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:50.744989 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46743a32-8d0a-4de5-a1a5-a138381b9baa None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:50.746644 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 99/193] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 15 07:19:50 2022] GET /identity/v3/projects/91757d436677474ab5f1027263238244 => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:19:50.747967 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46743a32-8d0a-4de5-a1a5-a138381b9baa None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:50.821612 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 95/194] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Sep 15 07:19:50 2022] POST /identity/v3/users => generated 297 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:19:50.821612 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb5999f-95cf-4eef-b972-ee67face029f None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:50.865871 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46743a32-8d0a-4de5-a1a5-a138381b9baa None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:50.882738 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:50.905391 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd62a7b-eb37-466e-97e9-570cafac9001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:50.906424 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd62a7b-eb37-466e-97e9-570cafac9001 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:50.906754 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd62a7b-eb37-466e-97e9-570cafac9001 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:50.907131 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd62a7b-eb37-466e-97e9-570cafac9001 None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:50.908026 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd62a7b-eb37-466e-97e9-570cafac9001 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:19:50.908436 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd62a7b-eb37-466e-97e9-570cafac9001 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:50.910149 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd62a7b-eb37-466e-97e9-570cafac9001 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:50.921638 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 100/195] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 15 07:19:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:19:50.928420 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd62a7b-eb37-466e-97e9-570cafac9001 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:50.976694 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb5999f-95cf-4eef-b972-ee67face029f None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:51.025597 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:51.033885 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4da2d0d-5a77-4f18-813a-98c3012ba0e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:51.034264 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4da2d0d-5a77-4f18-813a-98c3012ba0e1 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:51.036426 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4da2d0d-5a77-4f18-813a-98c3012ba0e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:51.036426 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4da2d0d-5a77-4f18-813a-98c3012ba0e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:51.037407 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4da2d0d-5a77-4f18-813a-98c3012ba0e1 None admin] RBAC: Adding query filter params (role.id=6bbb643e0d8d4bc0a9b27c26edf59216, scope.project.id=91757d436677474ab5f1027263238244, user.id=053b642391194fa899b9eca12ae3cf99) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:19:51.037823 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4da2d0d-5a77-4f18-813a-98c3012ba0e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:51.039385 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4da2d0d-5a77-4f18-813a-98c3012ba0e1 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:51.048002 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 96/196] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 15 07:19:50 2022] GET /identity/v3/role_assignments?user.id=053b642391194fa899b9eca12ae3cf99&scope.project.id=91757d436677474ab5f1027263238244&role.id=6bbb643e0d8d4bc0a9b27c26edf59216 => generated 259 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:19:51.077067 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd62a7b-eb37-466e-97e9-570cafac9001 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:51.117119 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf6083c-8cea-4f6a-9c25-75d18ba3f5bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:51.117886 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf6083c-8cea-4f6a-9c25-75d18ba3f5bf None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:51.118169 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf6083c-8cea-4f6a-9c25-75d18ba3f5bf None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:51.118442 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf6083c-8cea-4f6a-9c25-75d18ba3f5bf None admin] PATH_INFO: `/v3/users/053b642391194fa899b9eca12ae3cf99` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:51.119717 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf6083c-8cea-4f6a-9c25-75d18ba3f5bf None admin] RBAC: Authorizing `identity:get_user(user_id=053b642391194fa899b9eca12ae3cf99)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:51.121216 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf6083c-8cea-4f6a-9c25-75d18ba3f5bf None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:51.123703 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 101/197] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 15 07:19:50 2022] GET /identity/v3/users/053b642391194fa899b9eca12ae3cf99 => generated 283 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:19:51.129288 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4da2d0d-5a77-4f18-813a-98c3012ba0e1 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:51.253109 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4da2d0d-5a77-4f18-813a-98c3012ba0e1 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:51.289982 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00b1620-19e1-431d-8c65-17c6a2398c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:51.290962 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00b1620-19e1-431d-8c65-17c6a2398c83 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:51.291556 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00b1620-19e1-431d-8c65-17c6a2398c83 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:51.292035 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00b1620-19e1-431d-8c65-17c6a2398c83 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:51.366942 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:51.373658 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00b1620-19e1-431d-8c65-17c6a2398c83 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:51.375152 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00b1620-19e1-431d-8c65-17c6a2398c83 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:51.518586 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 97/198] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 15 07:19:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 07:19:51.524116 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf6083c-8cea-4f6a-9c25-75d18ba3f5bf None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:51.603525 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec32e5b-80f1-40cb-b725-dde7e63defb6 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:51.603858 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec32e5b-80f1-40cb-b725-dde7e63defb6 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:51.604080 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec32e5b-80f1-40cb-b725-dde7e63defb6 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:51.606202 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:51.606313 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 98/199] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:19:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 07:19:51.611576 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7452fbb5-c78c-47f8-8bec-aae9322f7de7 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:51.611855 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7452fbb5-c78c-47f8-8bec-aae9322f7de7 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:51.612131 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7452fbb5-c78c-47f8-8bec-aae9322f7de7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:51.652583 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-7452fbb5-c78c-47f8-8bec-aae9322f7de7 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:19:51.660002 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-7452fbb5-c78c-47f8-8bec-aae9322f7de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:19:51.714466 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf6083c-8cea-4f6a-9c25-75d18ba3f5bf None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:51.764400 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 99/200] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:19:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 07:19:51.792689 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f9fdcb-210a-4fbe-b9aa-34b61d8ab551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:51.793678 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f9fdcb-210a-4fbe-b9aa-34b61d8ab551 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:51.794194 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f9fdcb-210a-4fbe-b9aa-34b61d8ab551 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:51.794815 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f9fdcb-210a-4fbe-b9aa-34b61d8ab551 None admin] PATH_INFO: `/v3/projects/91757d436677474ab5f1027263238244` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:51.802580 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f9fdcb-210a-4fbe-b9aa-34b61d8ab551 None admin] RBAC: Authorizing `identity:get_project(project_id=91757d436677474ab5f1027263238244)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:51.804269 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f9fdcb-210a-4fbe-b9aa-34b61d8ab551 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:51.806406 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 102/201] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 15 07:19:51 2022] GET /identity/v3/projects/91757d436677474ab5f1027263238244 => generated 302 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:19:51.814005 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7452fbb5-c78c-47f8-8bec-aae9322f7de7 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:51.854267 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:51.933314 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7452fbb5-c78c-47f8-8bec-aae9322f7de7 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:51.992527 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84c4250-2026-4c0c-8cc0-16f19a807ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:51.993214 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84c4250-2026-4c0c-8cc0-16f19a807ac0 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:51.993500 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84c4250-2026-4c0c-8cc0-16f19a807ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:51.993799 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84c4250-2026-4c0c-8cc0-16f19a807ac0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:51.994543 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84c4250-2026-4c0c-8cc0-16f19a807ac0 None admin] RBAC: Adding query filter params (role.id=a0fae926490542b4977d1ecf27fcece4, scope.project.id=91757d436677474ab5f1027263238244, user.id=053b642391194fa899b9eca12ae3cf99) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:19:51.994851 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84c4250-2026-4c0c-8cc0-16f19a807ac0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:51.996293 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84c4250-2026-4c0c-8cc0-16f19a807ac0 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:52.003137 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 100/202] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 15 07:19:51 2022] GET /identity/v3/role_assignments?user.id=053b642391194fa899b9eca12ae3cf99&scope.project.id=91757d436677474ab5f1027263238244&role.id=a0fae926490542b4977d1ecf27fcece4 => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:19:52.096176 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f9fdcb-210a-4fbe-b9aa-34b61d8ab551 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:52.284858 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f9fdcb-210a-4fbe-b9aa-34b61d8ab551 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:52.304400 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:52.336021 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74baf26-a5d4-4ea1-bd0d-3967f583f27a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:52.337019 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74baf26-a5d4-4ea1-bd0d-3967f583f27a None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:52.337517 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74baf26-a5d4-4ea1-bd0d-3967f583f27a None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:52.338019 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74baf26-a5d4-4ea1-bd0d-3967f583f27a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:52.354938 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74baf26-a5d4-4ea1-bd0d-3967f583f27a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:19:52.361086 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-f74baf26-a5d4-4ea1-bd0d-3967f583f27a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 07:19:52.361086 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:19:52.361086 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:19:52.361086 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:19:52.361086 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:19:52.361086 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:19:52.361086 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:19:52.361086 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:19:52.361086 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:19:52.361086 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:19:52.361086 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:19:52.361086 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:19:52.361086 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:19:52.361086 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:19:52.361086 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:19:52.361086 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:19:52.361975 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:19:52.361975 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:19:52.361975 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:19:52.361975 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:19:52.361975 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:19:52.361975 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:19:52.361975 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:19:52.361975 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:19:52.361975 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:19:52.361975 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:19:52.361975 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:19:52.361975 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:19:52.361975 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:19:52.361975 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:19:52.361975 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:19:52.361975 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:19:52.362760 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:19:52.362760 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:19:52.362760 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:19:52.362760 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:19:52.362760 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:19:52.362760 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:19:52.362760 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:19:52.362760 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:19:52.362760 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:19:52.362760 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:19:52.362760 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 07:19:52.362760 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:19:52.365032 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 103/203] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 15 07:19:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:19:52.374998 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84c4250-2026-4c0c-8cc0-16f19a807ac0 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:52.503187 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:52.516028 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84c4250-2026-4c0c-8cc0-16f19a807ac0 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:52.563017 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110089b7-3cd5-4c5b-ab1a-c54430ac96be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:52.563641 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110089b7-3cd5-4c5b-ab1a-c54430ac96be None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:52.563925 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110089b7-3cd5-4c5b-ab1a-c54430ac96be None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:52.564200 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110089b7-3cd5-4c5b-ab1a-c54430ac96be None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:52.564869 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110089b7-3cd5-4c5b-ab1a-c54430ac96be None admin] RBAC: Adding query filter params (name=reader) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:19:52.565176 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110089b7-3cd5-4c5b-ab1a-c54430ac96be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:52.566593 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110089b7-3cd5-4c5b-ab1a-c54430ac96be None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:52.583311 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 101/204] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Sep 15 07:19:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:19:52.589258 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74baf26-a5d4-4ea1-bd0d-3967f583f27a None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:52.735040 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74baf26-a5d4-4ea1-bd0d-3967f583f27a None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:52.780698 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9f30d5-261b-4499-9d4d-c2a7314be0b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:52.781721 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9f30d5-261b-4499-9d4d-c2a7314be0b3 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:52.782220 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9f30d5-261b-4499-9d4d-c2a7314be0b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:52.782695 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9f30d5-261b-4499-9d4d-c2a7314be0b3 None admin] PATH_INFO: `/v3/users/7fdb3deb1979491fa3690c7fdf6cc63b` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:52.784036 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9f30d5-261b-4499-9d4d-c2a7314be0b3 None admin] RBAC: Authorizing `identity:get_user(user_id=7fdb3deb1979491fa3690c7fdf6cc63b)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:52.785822 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9f30d5-261b-4499-9d4d-c2a7314be0b3 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:52.787664 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 104/205] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 15 07:19:52 2022] GET /identity/v3/users/7fdb3deb1979491fa3690c7fdf6cc63b => generated 289 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:19:52.793547 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110089b7-3cd5-4c5b-ab1a-c54430ac96be None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:52.945179 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110089b7-3cd5-4c5b-ab1a-c54430ac96be None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:53.004451 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9b42e7-dc33-43c8-9ecf-6332c097acb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:53.005290 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9b42e7-dc33-43c8-9ecf-6332c097acb7 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:53.005763 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9b42e7-dc33-43c8-9ecf-6332c097acb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:53.006181 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9b42e7-dc33-43c8-9ecf-6332c097acb7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:53.041600 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:53.103619 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9b42e7-dc33-43c8-9ecf-6332c097acb7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:53.105305 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9b42e7-dc33-43c8-9ecf-6332c097acb7 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:53.273278 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 102/206] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 15 07:19:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 07:19:53.279187 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9f30d5-261b-4499-9d4d-c2a7314be0b3 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:53.355983 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:53.465616 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9f30d5-261b-4499-9d4d-c2a7314be0b3 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:53.506594 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccaf980-0e84-43cd-9142-2ccde16f315d None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:53.506910 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccaf980-0e84-43cd-9142-2ccde16f315d None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:53.507159 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccaf980-0e84-43cd-9142-2ccde16f315d None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:53.508530 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 103/207] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:19:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 07:19:53.516621 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c76bb0e-a178-41ff-b498-d9d31dd8f47b None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:53.516908 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c76bb0e-a178-41ff-b498-d9d31dd8f47b None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:53.517186 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c76bb0e-a178-41ff-b498-d9d31dd8f47b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:53.545980 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-4c76bb0e-a178-41ff-b498-d9d31dd8f47b None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:19:53.547515 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0a92da-79f3-4e56-be4a-e1d6201927b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:53.547644 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-4c76bb0e-a178-41ff-b498-d9d31dd8f47b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:19:53.548175 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0a92da-79f3-4e56-be4a-e1d6201927b2 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:53.548454 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0a92da-79f3-4e56-be4a-e1d6201927b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:53.548726 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0a92da-79f3-4e56-be4a-e1d6201927b2 None admin] PATH_INFO: `/v3/projects/659ee9038adf47e585b94881867c18c2` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:53.551952 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0a92da-79f3-4e56-be4a-e1d6201927b2 None admin] RBAC: Authorizing `identity:get_project(project_id=659ee9038adf47e585b94881867c18c2)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:53.553888 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0a92da-79f3-4e56-be4a-e1d6201927b2 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:53.556132 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 105/208] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 15 07:19:53 2022] GET /identity/v3/projects/659ee9038adf47e585b94881867c18c2 => generated 298 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:19:53.562601 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0a92da-79f3-4e56-be4a-e1d6201927b2 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:53.629953 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:53.681669 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 104/209] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:19:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 07:19:53.718415 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0a92da-79f3-4e56-be4a-e1d6201927b2 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:53.735640 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:53.777151 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46c5a06-a023-4752-bbf5-a1f12408a47f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:53.778155 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46c5a06-a023-4752-bbf5-a1f12408a47f None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:53.778727 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46c5a06-a023-4752-bbf5-a1f12408a47f None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:53.779847 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46c5a06-a023-4752-bbf5-a1f12408a47f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:53.780772 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46c5a06-a023-4752-bbf5-a1f12408a47f None admin] RBAC: Adding query filter params (role.id=81b8a5256ba14184beb300947fbfe335, scope.project.id=659ee9038adf47e585b94881867c18c2, user.id=7fdb3deb1979491fa3690c7fdf6cc63b) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:19:53.781283 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46c5a06-a023-4752-bbf5-a1f12408a47f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:53.783041 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46c5a06-a023-4752-bbf5-a1f12408a47f None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:53.796692 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 106/210] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 15 07:19:53 2022] GET /identity/v3/role_assignments?user.id=7fdb3deb1979491fa3690c7fdf6cc63b&scope.project.id=659ee9038adf47e585b94881867c18c2&role.id=81b8a5256ba14184beb300947fbfe335 => generated 608 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:19:53.952414 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffbb144-d957-4223-8bc7-d1d9636b9552 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:53.952911 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffbb144-d957-4223-8bc7-d1d9636b9552 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:53.952983 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffbb144-d957-4223-8bc7-d1d9636b9552 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:53.954600 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 105/211] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:19:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:19:53.959954 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1c8246-0b97-4c31-83e4-8b3fb9213e4d None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:53.960549 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1c8246-0b97-4c31-83e4-8b3fb9213e4d None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:53.960976 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffbb144-d957-4223-8bc7-d1d9636b9552 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:53.961236 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1c8246-0b97-4c31-83e4-8b3fb9213e4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:53.985928 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-9a1c8246-0b97-4c31-83e4-8b3fb9213e4d None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:19:53.987596 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-9a1c8246-0b97-4c31-83e4-8b3fb9213e4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:19:54.090679 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 107/212] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:19:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 07:19:54.093802 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04797350-edcc-41bf-a00b-0ab225a728a5 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:54.094094 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04797350-edcc-41bf-a00b-0ab225a728a5 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:54.094369 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04797350-edcc-41bf-a00b-0ab225a728a5 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:54.095746 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 108/213] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:19:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:19:54.101126 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615e2f00-3193-441b-b7db-238e1f95c590 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:54.101410 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615e2f00-3193-441b-b7db-238e1f95c590 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:54.101728 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615e2f00-3193-441b-b7db-238e1f95c590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:54.114496 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffbb144-d957-4223-8bc7-d1d9636b9552 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:54.127416 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-615e2f00-3193-441b-b7db-238e1f95c590 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:19:54.128322 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-615e2f00-3193-441b-b7db-238e1f95c590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:19:54.157668 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839e41c0-41aa-44d8-a07c-c431989afdce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:54.166962 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839e41c0-41aa-44d8-a07c-c431989afdce None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:54.166962 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839e41c0-41aa-44d8-a07c-c431989afdce None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:54.166962 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839e41c0-41aa-44d8-a07c-c431989afdce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:54.178918 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839e41c0-41aa-44d8-a07c-c431989afdce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:19:54.186781 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-839e41c0-41aa-44d8-a07c-c431989afdce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 07:19:54.186781 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:19:54.186781 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:19:54.186781 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:19:54.186781 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:19:54.186781 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:19:54.186781 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:19:54.186781 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:19:54.186781 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:19:54.186781 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:19:54.186781 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:19:54.186781 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:19:54.186781 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:19:54.186781 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:19:54.186781 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:19:54.186781 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:19:54.187695 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:19:54.187695 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:19:54.187695 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:19:54.187695 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:19:54.187695 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:19:54.187695 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:19:54.187695 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:19:54.187695 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:19:54.187695 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:19:54.187695 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:19:54.187695 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:19:54.187695 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:19:54.187695 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:19:54.187695 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:19:54.187695 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:19:54.187695 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:19:54.188457 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:19:54.188457 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:19:54.188457 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:19:54.188457 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:19:54.188457 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:19:54.188457 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:19:54.188457 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:19:54.188457 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:19:54.188457 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:19:54.188457 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:19:54.188457 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 07:19:54.188457 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:19:54.191352 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 106/214] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 15 07:19:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:19:54.196669 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839e41c0-41aa-44d8-a07c-c431989afdce None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:54.225980 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 109/215] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:19:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 07:19:54.257092 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:54.320108 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839e41c0-41aa-44d8-a07c-c431989afdce None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:54.356177 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bc65b5-46b5-49ef-8885-dafc1979b25e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:54.356709 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bc65b5-46b5-49ef-8885-dafc1979b25e None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:54.357981 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bc65b5-46b5-49ef-8885-dafc1979b25e None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:54.357981 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bc65b5-46b5-49ef-8885-dafc1979b25e None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:54.358690 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bc65b5-46b5-49ef-8885-dafc1979b25e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:19:54.358999 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bc65b5-46b5-49ef-8885-dafc1979b25e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:54.360372 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bc65b5-46b5-49ef-8885-dafc1979b25e None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:54.369301 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 107/216] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 15 07:19:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:19:54.374585 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615e2f00-3193-441b-b7db-238e1f95c590 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:54.403594 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bc65b5-46b5-49ef-8885-dafc1979b25e None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:54.411379 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:54.492560 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615e2f00-3193-441b-b7db-238e1f95c590 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:54.536098 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cf51d1-709d-4279-91b9-de4893a6e418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:54.536844 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cf51d1-709d-4279-91b9-de4893a6e418 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:54.537130 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cf51d1-709d-4279-91b9-de4893a6e418 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:54.537407 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cf51d1-709d-4279-91b9-de4893a6e418 None admin] PATH_INFO: `/v3/users/053b642391194fa899b9eca12ae3cf99` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:54.538576 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cf51d1-709d-4279-91b9-de4893a6e418 None admin] RBAC: Authorizing `identity:get_user(user_id=053b642391194fa899b9eca12ae3cf99)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:54.540237 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cf51d1-709d-4279-91b9-de4893a6e418 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:54.542154 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 110/217] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 15 07:19:54 2022] GET /identity/v3/users/053b642391194fa899b9eca12ae3cf99 => generated 283 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:19:54.542564 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bc65b5-46b5-49ef-8885-dafc1979b25e None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:54.549640 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:54.549786 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cf51d1-709d-4279-91b9-de4893a6e418 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:54.585853 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d196e2-220a-4ecc-b92f-5ca08daf64b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:54.586510 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d196e2-220a-4ecc-b92f-5ca08daf64b4 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:54.587180 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d196e2-220a-4ecc-b92f-5ca08daf64b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:54.588479 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d196e2-220a-4ecc-b92f-5ca08daf64b4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:54.589276 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d196e2-220a-4ecc-b92f-5ca08daf64b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:54.590752 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d196e2-220a-4ecc-b92f-5ca08daf64b4 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:54.592715 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 108/218] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:19:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:19:54.597265 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d196e2-220a-4ecc-b92f-5ca08daf64b4 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:54.660590 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:54.663508 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cf51d1-709d-4279-91b9-de4893a6e418 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:54.702837 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecc55cf-0c06-4cf8-80cf-7842695e21a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:54.703464 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecc55cf-0c06-4cf8-80cf-7842695e21a8 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:54.703741 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecc55cf-0c06-4cf8-80cf-7842695e21a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:54.704069 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecc55cf-0c06-4cf8-80cf-7842695e21a8 None admin] PATH_INFO: `/v3/projects/91757d436677474ab5f1027263238244` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:54.705182 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecc55cf-0c06-4cf8-80cf-7842695e21a8 None admin] RBAC: Authorizing `identity:get_project(project_id=91757d436677474ab5f1027263238244)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:54.706664 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecc55cf-0c06-4cf8-80cf-7842695e21a8 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:54.713240 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d196e2-220a-4ecc-b92f-5ca08daf64b4 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:54.713370 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 111/219] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 15 07:19:54 2022] GET /identity/v3/projects/91757d436677474ab5f1027263238244 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:19:54.713370 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecc55cf-0c06-4cf8-80cf-7842695e21a8 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:54.748650 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523b39d4-39c4-440d-a843-2a388e9ca4d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:54.749295 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523b39d4-39c4-440d-a843-2a388e9ca4d4 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:54.749610 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523b39d4-39c4-440d-a843-2a388e9ca4d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:54.749890 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523b39d4-39c4-440d-a843-2a388e9ca4d4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:54.760459 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523b39d4-39c4-440d-a843-2a388e9ca4d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:19:54.765456 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-523b39d4-39c4-440d-a843-2a388e9ca4d4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 07:19:54.765456 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:19:54.765456 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:19:54.765456 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:19:54.765456 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:19:54.765456 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:19:54.765456 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:19:54.765456 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:19:54.765456 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:19:54.765456 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:19:54.765456 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:19:54.765456 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:19:54.765456 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:19:54.765456 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:19:54.765456 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:19:54.765456 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:19:54.766379 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:19:54.766379 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:19:54.766379 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:19:54.766379 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:19:54.766379 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:19:54.766379 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:19:54.766379 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:19:54.766379 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:19:54.766379 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:19:54.766379 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:19:54.766379 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:19:54.766379 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:19:54.766379 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:19:54.766379 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:19:54.766379 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:19:54.766379 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:19:54.770432 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:19:54.770432 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:19:54.770432 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:19:54.770432 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:19:54.770432 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:19:54.770432 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:19:54.770432 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:19:54.770432 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:19:54.770432 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:19:54.770432 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:19:54.770432 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 07:19:54.770432 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:19:54.770432 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 109/220] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 15 07:19:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:19:54.770432 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523b39d4-39c4-440d-a843-2a388e9ca4d4 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:54.802721 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:54.830252 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecc55cf-0c06-4cf8-80cf-7842695e21a8 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:54.868408 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75e5350-a5ed-4ba5-80dd-b7009ed87bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:54.869013 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75e5350-a5ed-4ba5-80dd-b7009ed87bff None admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:54.869288 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75e5350-a5ed-4ba5-80dd-b7009ed87bff None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:54.869605 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75e5350-a5ed-4ba5-80dd-b7009ed87bff None admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:54.870223 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75e5350-a5ed-4ba5-80dd-b7009ed87bff None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:54.871671 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75e5350-a5ed-4ba5-80dd-b7009ed87bff None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:54.887427 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523b39d4-39c4-440d-a843-2a388e9ca4d4 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:54.922621 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2b29f3-37dd-41a2-87d0-e5a949732dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:54.923256 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2b29f3-37dd-41a2-87d0-e5a949732dcd None admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:54.923532 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2b29f3-37dd-41a2-87d0-e5a949732dcd None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:54.923807 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2b29f3-37dd-41a2-87d0-e5a949732dcd None admin] PATH_INFO: `/v3/projects/91757d436677474ab5f1027263238244/users/053b642391194fa899b9eca12ae3cf99/roles/6bbb643e0d8d4bc0a9b27c26edf59216` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:54.925770 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2b29f3-37dd-41a2-87d0-e5a949732dcd None admin] RBAC: Authorizing `identity:create_grant(project_id=91757d436677474ab5f1027263238244, user_id=053b642391194fa899b9eca12ae3cf99, role_id=6bbb643e0d8d4bc0a9b27c26edf59216)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:54.927279 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2b29f3-37dd-41a2-87d0-e5a949732dcd None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:54.964748 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 112/221] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Sep 15 07:19:54 2022] POST /identity/v3/users => generated 297 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:19:54.965786 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75e5350-a5ed-4ba5-80dd-b7009ed87bff None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:54.994604 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:55.028324 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 110/222] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Sep 15 07:19:54 2022] PUT /identity/v3/projects/91757d436677474ab5f1027263238244/users/053b642391194fa899b9eca12ae3cf99/roles/6bbb643e0d8d4bc0a9b27c26edf59216 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:19:55.039235 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb55efc2-51ab-4167-a954-096ffe5dafbd None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:55.039530 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb55efc2-51ab-4167-a954-096ffe5dafbd None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:55.039786 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb55efc2-51ab-4167-a954-096ffe5dafbd None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:55.041336 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 111/223] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:19:55 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:19:55.047867 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061715b6-e3aa-462e-9283-68e053358863 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:55.048181 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061715b6-e3aa-462e-9283-68e053358863 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:55.048431 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061715b6-e3aa-462e-9283-68e053358863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:55.085439 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-061715b6-e3aa-462e-9283-68e053358863 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:19:55.087282 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-061715b6-e3aa-462e-9283-68e053358863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:19:55.088402 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75e5350-a5ed-4ba5-80dd-b7009ed87bff None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:55.134582 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7aae3d-f77f-4446-a5c2-2845bac16f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:55.135329 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7aae3d-f77f-4446-a5c2-2845bac16f51 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:55.135618 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7aae3d-f77f-4446-a5c2-2845bac16f51 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:55.135902 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7aae3d-f77f-4446-a5c2-2845bac16f51 None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:55.136593 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7aae3d-f77f-4446-a5c2-2845bac16f51 None admin] RBAC: Adding query filter params (name=member) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:19:55.136901 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7aae3d-f77f-4446-a5c2-2845bac16f51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:55.139258 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7aae3d-f77f-4446-a5c2-2845bac16f51 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:55.150213 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 113/224] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Sep 15 07:19:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:19:55.157013 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7aae3d-f77f-4446-a5c2-2845bac16f51 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:55.191609 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 112/225] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:19:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 07:19:55.259167 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:55.275104 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7aae3d-f77f-4446-a5c2-2845bac16f51 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:55.311749 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217dc33a-149e-43e0-af6f-57fc823be429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:55.313094 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217dc33a-149e-43e0-af6f-57fc823be429 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:55.313381 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217dc33a-149e-43e0-af6f-57fc823be429 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:55.313687 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217dc33a-149e-43e0-af6f-57fc823be429 None admin] PATH_INFO: `/v3/users/0339c9cb36724417a2680476b09e7033` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:55.331736 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217dc33a-149e-43e0-af6f-57fc823be429 None admin] RBAC: Authorizing `identity:get_user(user_id=0339c9cb36724417a2680476b09e7033)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:55.333215 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217dc33a-149e-43e0-af6f-57fc823be429 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:55.335160 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 114/226] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 15 07:19:55 2022] GET /identity/v3/users/0339c9cb36724417a2680476b09e7033 => generated 297 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:19:55.340518 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061715b6-e3aa-462e-9283-68e053358863 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:55.376169 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:55.471336 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061715b6-e3aa-462e-9283-68e053358863 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:55.480712 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217dc33a-149e-43e0-af6f-57fc823be429 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:55.515427 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23df88bc-5d61-4159-b52a-fefd13504d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:55.516414 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23df88bc-5d61-4159-b52a-fefd13504d14 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:55.516827 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23df88bc-5d61-4159-b52a-fefd13504d14 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:55.517250 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23df88bc-5d61-4159-b52a-fefd13504d14 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:55.606932 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23df88bc-5d61-4159-b52a-fefd13504d14 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:55.608380 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23df88bc-5d61-4159-b52a-fefd13504d14 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:55.630866 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217dc33a-149e-43e0-af6f-57fc823be429 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:55.654905 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:55.695106 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892220c5-efe3-43ff-ba85-a04901432d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:55.695754 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892220c5-efe3-43ff-ba85-a04901432d90 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:55.696029 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892220c5-efe3-43ff-ba85-a04901432d90 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:55.696300 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892220c5-efe3-43ff-ba85-a04901432d90 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:55.715270 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892220c5-efe3-43ff-ba85-a04901432d90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:19:55.720778 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-892220c5-efe3-43ff-ba85-a04901432d90 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 07:19:55.720778 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:19:55.720778 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:19:55.720778 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:19:55.720778 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:19:55.720778 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:19:55.720778 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:19:55.720778 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:19:55.720778 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:19:55.720778 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:19:55.720778 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:19:55.720778 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:19:55.720778 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:19:55.720778 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:19:55.720778 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:19:55.720778 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:19:55.721730 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:19:55.721730 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:19:55.721730 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:19:55.721730 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:19:55.721730 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:19:55.721730 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:19:55.721730 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:19:55.721730 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:19:55.721730 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:19:55.721730 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:19:55.721730 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:19:55.721730 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:19:55.721730 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:19:55.721730 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:19:55.721730 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:19:55.721730 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:19:55.722517 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:19:55.722517 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:19:55.722517 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:19:55.722517 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:19:55.722517 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:19:55.722517 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:19:55.722517 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:19:55.722517 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:19:55.722517 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:19:55.722517 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:19:55.722517 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 07:19:55.722517 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:19:55.722517 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 115/227] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Sep 15 07:19:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 07:19:55.731750 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892220c5-efe3-43ff-ba85-a04901432d90 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:55.745924 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:55.799652 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 113/228] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 15 07:19:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 07:19:55.807276 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23df88bc-5d61-4159-b52a-fefd13504d14 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:55.939349 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892220c5-efe3-43ff-ba85-a04901432d90 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:55.992164 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23df88bc-5d61-4159-b52a-fefd13504d14 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:56.011612 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04f406b-00a7-4288-840e-d478a82b58df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:56.012235 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04f406b-00a7-4288-840e-d478a82b58df None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:56.012513 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04f406b-00a7-4288-840e-d478a82b58df None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:56.013620 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04f406b-00a7-4288-840e-d478a82b58df None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:56.014276 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04f406b-00a7-4288-840e-d478a82b58df None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:19:56.014580 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04f406b-00a7-4288-840e-d478a82b58df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:56.015969 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04f406b-00a7-4288-840e-d478a82b58df None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:56.037716 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 116/229] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 15 07:19:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:19:56.040961 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:56.043548 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04f406b-00a7-4288-840e-d478a82b58df None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:56.066991 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0341c2b-4523-4579-a7b6-a4104ab5f86c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:56.068031 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0341c2b-4523-4579-a7b6-a4104ab5f86c None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:56.069712 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0341c2b-4523-4579-a7b6-a4104ab5f86c None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:56.070204 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0341c2b-4523-4579-a7b6-a4104ab5f86c None admin] PATH_INFO: `/v3/projects/91757d436677474ab5f1027263238244` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:56.072215 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0341c2b-4523-4579-a7b6-a4104ab5f86c None admin] RBAC: Authorizing `identity:get_project(project_id=91757d436677474ab5f1027263238244)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:56.074111 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0341c2b-4523-4579-a7b6-a4104ab5f86c None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:56.078059 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 114/230] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 15 07:19:55 2022] GET /identity/v3/projects/91757d436677474ab5f1027263238244 => generated 302 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:19:56.086027 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0341c2b-4523-4579-a7b6-a4104ab5f86c None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:56.164652 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:56.218988 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04f406b-00a7-4288-840e-d478a82b58df None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:56.278384 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0341c2b-4523-4579-a7b6-a4104ab5f86c None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:56.282606 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306421ce-d415-445d-9ce8-b4ad90bb7798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:56.283257 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306421ce-d415-445d-9ce8-b4ad90bb7798 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:56.283532 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306421ce-d415-445d-9ce8-b4ad90bb7798 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:56.283900 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306421ce-d415-445d-9ce8-b4ad90bb7798 None admin] PATH_INFO: `/v3/users/053b642391194fa899b9eca12ae3cf99` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:56.285803 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306421ce-d415-445d-9ce8-b4ad90bb7798 None admin] RBAC: Authorizing `identity:get_user(user_id=053b642391194fa899b9eca12ae3cf99)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:56.287277 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306421ce-d415-445d-9ce8-b4ad90bb7798 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:56.288882 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 117/231] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 15 07:19:56 2022] GET /identity/v3/users/053b642391194fa899b9eca12ae3cf99 => generated 283 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:19:56.297790 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306421ce-d415-445d-9ce8-b4ad90bb7798 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:56.346726 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80cff62-aa0e-4ce8-a8e3-d5504be28926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:56.347382 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80cff62-aa0e-4ce8-a8e3-d5504be28926 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:56.347665 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80cff62-aa0e-4ce8-a8e3-d5504be28926 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:56.347935 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80cff62-aa0e-4ce8-a8e3-d5504be28926 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:56.348655 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80cff62-aa0e-4ce8-a8e3-d5504be28926 None admin] RBAC: Adding query filter params (role.id=053e12af84e943e699460a9d76c2072b, scope.project.id=91757d436677474ab5f1027263238244, user.id=0339c9cb36724417a2680476b09e7033) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:19:56.348960 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80cff62-aa0e-4ce8-a8e3-d5504be28926 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:56.350411 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80cff62-aa0e-4ce8-a8e3-d5504be28926 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:56.361191 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 115/232] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 15 07:19:56 2022] GET /identity/v3/role_assignments?user.id=0339c9cb36724417a2680476b09e7033&scope.project.id=91757d436677474ab5f1027263238244&role.id=053e12af84e943e699460a9d76c2072b => generated 259 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:19:56.466110 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306421ce-d415-445d-9ce8-b4ad90bb7798 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:56.522190 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86995fc-4d55-40b2-a0e4-3123e4b59d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:56.522929 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86995fc-4d55-40b2-a0e4-3123e4b59d89 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:56.523172 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86995fc-4d55-40b2-a0e4-3123e4b59d89 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:56.523448 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86995fc-4d55-40b2-a0e4-3123e4b59d89 None admin] PATH_INFO: `/v3/projects/91757d436677474ab5f1027263238244` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:56.527509 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86995fc-4d55-40b2-a0e4-3123e4b59d89 None admin] RBAC: Authorizing `identity:get_project(project_id=91757d436677474ab5f1027263238244)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:56.529628 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86995fc-4d55-40b2-a0e4-3123e4b59d89 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:56.531447 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 118/233] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 15 07:19:56 2022] GET /identity/v3/projects/91757d436677474ab5f1027263238244 => generated 302 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:19:56.538840 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80cff62-aa0e-4ce8-a8e3-d5504be28926 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:56.600096 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:56.646946 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80cff62-aa0e-4ce8-a8e3-d5504be28926 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:56.681909 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab9f1c5-4fb6-407d-a303-593bb703a668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:56.683803 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab9f1c5-4fb6-407d-a303-593bb703a668 None admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:56.684083 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab9f1c5-4fb6-407d-a303-593bb703a668 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:56.684356 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab9f1c5-4fb6-407d-a303-593bb703a668 None admin] PATH_INFO: `/v3/projects/91757d436677474ab5f1027263238244/users/053b642391194fa899b9eca12ae3cf99/roles/a0fae926490542b4977d1ecf27fcece4` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:56.686717 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab9f1c5-4fb6-407d-a303-593bb703a668 None admin] RBAC: Authorizing `identity:create_grant(project_id=91757d436677474ab5f1027263238244, user_id=053b642391194fa899b9eca12ae3cf99, role_id=a0fae926490542b4977d1ecf27fcece4)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:56.688148 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab9f1c5-4fb6-407d-a303-593bb703a668 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:56.793903 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 116/234] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Sep 15 07:19:56 2022] PUT /identity/v3/projects/91757d436677474ab5f1027263238244/users/053b642391194fa899b9eca12ae3cf99/roles/a0fae926490542b4977d1ecf27fcece4 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:19:57.491614 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ab5818-67a6-4831-9f26-928a79ae6f1c None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:57.491983 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ab5818-67a6-4831-9f26-928a79ae6f1c None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:57.492145 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ab5818-67a6-4831-9f26-928a79ae6f1c None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:57.493790 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 119/235] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:19:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:19:57.499331 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fc5334-a12d-42eb-b9ac-c80d6dca7ccf None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:57.499892 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fc5334-a12d-42eb-b9ac-c80d6dca7ccf None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:57.499892 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fc5334-a12d-42eb-b9ac-c80d6dca7ccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:57.524452 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-e6fc5334-a12d-42eb-b9ac-c80d6dca7ccf None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:19:57.525663 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-e6fc5334-a12d-42eb-b9ac-c80d6dca7ccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:19:57.619981 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 117/236] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:19:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 07:19:57.664283 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc5e4e-89b0-4dfa-b635-622254717843 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:57.664922 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc5e4e-89b0-4dfa-b635-622254717843 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:57.664922 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc5e4e-89b0-4dfa-b635-622254717843 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:57.666445 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 120/237] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:19:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:19:57.672442 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97489d2b-49b4-4cd9-9c05-7db80821f36d None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:57.672728 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97489d2b-49b4-4cd9-9c05-7db80821f36d None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:57.673004 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97489d2b-49b4-4cd9-9c05-7db80821f36d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:57.693044 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-97489d2b-49b4-4cd9-9c05-7db80821f36d None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:19:57.694294 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-97489d2b-49b4-4cd9-9c05-7db80821f36d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:19:57.737428 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:57.765729 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 118/238] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:19:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 07:19:57.984940 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f0502b-64f3-43a9-976b-c824dddcf37c None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:57.985567 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f0502b-64f3-43a9-976b-c824dddcf37c None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:57.986084 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f0502b-64f3-43a9-976b-c824dddcf37c None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:57.987655 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 121/239] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:19:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:19:57.993139 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffd7eb7-8e34-4877-8df7-886e9031d747 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:57.993430 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffd7eb7-8e34-4877-8df7-886e9031d747 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:57.993730 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffd7eb7-8e34-4877-8df7-886e9031d747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:58.021892 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-bffd7eb7-8e34-4877-8df7-886e9031d747 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:19:58.022990 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-bffd7eb7-8e34-4877-8df7-886e9031d747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:19:58.029344 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f0502b-64f3-43a9-976b-c824dddcf37c None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:58.036960 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:58.092821 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 119/240] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:19:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 07:19:58.093709 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffd7eb7-8e34-4877-8df7-886e9031d747 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:58.191195 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f0502b-64f3-43a9-976b-c824dddcf37c None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:58.194297 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffd7eb7-8e34-4877-8df7-886e9031d747 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:58.200372 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:58.230666 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a897d70-c026-4054-832a-f5e4c9b6893a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:58.231310 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a897d70-c026-4054-832a-f5e4c9b6893a None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:58.231593 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a897d70-c026-4054-832a-f5e4c9b6893a None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:58.231865 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a897d70-c026-4054-832a-f5e4c9b6893a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:58.243662 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a897d70-c026-4054-832a-f5e4c9b6893a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:19:58.249286 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-3a897d70-c026-4054-832a-f5e4c9b6893a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 07:19:58.249286 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:19:58.249286 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:19:58.249286 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:19:58.249286 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:19:58.249286 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:19:58.249286 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:19:58.249286 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:19:58.249286 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:19:58.249286 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:19:58.249286 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:19:58.249286 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:19:58.249286 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:19:58.249286 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:19:58.249286 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:19:58.249286 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:19:58.250813 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:19:58.250813 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:19:58.250813 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:19:58.250813 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:19:58.250813 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:19:58.250813 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:19:58.250813 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:19:58.250813 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:19:58.250813 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:19:58.250813 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:19:58.250813 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:19:58.250813 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:19:58.250813 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:19:58.250813 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:19:58.250813 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:19:58.250813 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:19:58.252136 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:19:58.252136 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:19:58.252136 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:19:58.252136 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:19:58.252136 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:19:58.252136 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:19:58.252136 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:19:58.252136 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:19:58.252136 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:19:58.252136 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:19:58.252136 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 07:19:58.252136 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:19:58.252136 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 120/241] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 15 07:19:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:19:58.261851 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a897d70-c026-4054-832a-f5e4c9b6893a None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:58.266337 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e45d9b8-64c4-40a2-9141-6699d530e97d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:58.267216 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e45d9b8-64c4-40a2-9141-6699d530e97d None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:58.267732 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e45d9b8-64c4-40a2-9141-6699d530e97d None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:58.268224 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e45d9b8-64c4-40a2-9141-6699d530e97d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:58.289818 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e45d9b8-64c4-40a2-9141-6699d530e97d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:19:58.296307 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-5e45d9b8-64c4-40a2-9141-6699d530e97d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 07:19:58.296307 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:19:58.296307 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:19:58.296307 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:19:58.296307 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:19:58.296307 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:19:58.296307 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:19:58.296307 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:19:58.296307 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:19:58.296307 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:19:58.296307 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:19:58.296307 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:19:58.296307 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:19:58.296307 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:19:58.296307 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:19:58.296307 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:19:58.297185 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:19:58.297185 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:19:58.297185 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:19:58.297185 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:19:58.297185 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:19:58.297185 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:19:58.297185 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:19:58.297185 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:19:58.297185 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:19:58.297185 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:19:58.297185 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:19:58.297185 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:19:58.297185 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:19:58.297185 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:19:58.297185 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:19:58.297185 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:19:58.298011 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:19:58.298011 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:19:58.298011 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:19:58.298011 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:19:58.298011 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:19:58.298011 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:19:58.298011 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:19:58.298011 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:19:58.298011 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:19:58.298011 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:19:58.298011 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 07:19:58.298011 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:19:58.311861 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 122/242] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 15 07:19:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:19:58.322559 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e45d9b8-64c4-40a2-9141-6699d530e97d None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:58.368458 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a897d70-c026-4054-832a-f5e4c9b6893a None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:58.426589 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79af1f91-059b-48aa-8dfb-416b1f441c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:58.427195 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79af1f91-059b-48aa-8dfb-416b1f441c55 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:58.427471 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79af1f91-059b-48aa-8dfb-416b1f441c55 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:58.427737 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79af1f91-059b-48aa-8dfb-416b1f441c55 None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:58.428406 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79af1f91-059b-48aa-8dfb-416b1f441c55 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:19:58.428706 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79af1f91-059b-48aa-8dfb-416b1f441c55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:58.430104 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79af1f91-059b-48aa-8dfb-416b1f441c55 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:58.452470 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 121/243] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Sep 15 07:19:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:19:58.460121 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79af1f91-059b-48aa-8dfb-416b1f441c55 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:58.476418 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:58.496269 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:58.527519 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e45d9b8-64c4-40a2-9141-6699d530e97d None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:58.578070 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79af1f91-059b-48aa-8dfb-416b1f441c55 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:58.578612 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adca9f3-1c62-4a7e-ae95-cf0b347ebf89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:58.579075 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adca9f3-1c62-4a7e-ae95-cf0b347ebf89 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:58.579352 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adca9f3-1c62-4a7e-ae95-cf0b347ebf89 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:58.579622 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adca9f3-1c62-4a7e-ae95-cf0b347ebf89 None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:58.580277 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adca9f3-1c62-4a7e-ae95-cf0b347ebf89 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:19:58.580575 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adca9f3-1c62-4a7e-ae95-cf0b347ebf89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:58.583128 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adca9f3-1c62-4a7e-ae95-cf0b347ebf89 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:58.593495 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 123/244] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 15 07:19:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:19:58.597679 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adca9f3-1c62-4a7e-ae95-cf0b347ebf89 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:58.622199 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0da8f8-0530-4a00-b8af-adb68fa8a1de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:58.623721 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0da8f8-0530-4a00-b8af-adb68fa8a1de None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:58.624197 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0da8f8-0530-4a00-b8af-adb68fa8a1de None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:58.624718 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0da8f8-0530-4a00-b8af-adb68fa8a1de None admin] PATH_INFO: `/v3/users/44cdef2ad9f740ef8e5cbde0b851738e` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:58.641082 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0da8f8-0530-4a00-b8af-adb68fa8a1de None admin] RBAC: Authorizing `identity:get_user(user_id=44cdef2ad9f740ef8e5cbde0b851738e)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:58.643036 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0da8f8-0530-4a00-b8af-adb68fa8a1de None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:58.645302 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 122/245] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 15 07:19:58 2022] GET /identity/v3/users/44cdef2ad9f740ef8e5cbde0b851738e => generated 297 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:19:58.646428 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0da8f8-0530-4a00-b8af-adb68fa8a1de None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:58.710424 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adca9f3-1c62-4a7e-ae95-cf0b347ebf89 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:58.734319 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:58.743374 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:58.745621 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49ae555-1741-4e65-ae4a-476db1f70b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:58.746109 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49ae555-1741-4e65-ae4a-476db1f70b10 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:58.746388 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49ae555-1741-4e65-ae4a-476db1f70b10 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:58.746658 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49ae555-1741-4e65-ae4a-476db1f70b10 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:58.748333 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49ae555-1741-4e65-ae4a-476db1f70b10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:58.749892 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49ae555-1741-4e65-ae4a-476db1f70b10 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:58.756046 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 124/246] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:19:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:19:58.756889 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49ae555-1741-4e65-ae4a-476db1f70b10 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:58.763833 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0da8f8-0530-4a00-b8af-adb68fa8a1de None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:58.802481 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c43e038-9885-4ccd-af2a-19625f981c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:58.803945 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c43e038-9885-4ccd-af2a-19625f981c8b None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:58.804529 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c43e038-9885-4ccd-af2a-19625f981c8b None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:58.805000 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c43e038-9885-4ccd-af2a-19625f981c8b None admin] PATH_INFO: `/v3/users/053b642391194fa899b9eca12ae3cf99` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:58.806753 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c43e038-9885-4ccd-af2a-19625f981c8b None admin] RBAC: Authorizing `identity:get_user(user_id=053b642391194fa899b9eca12ae3cf99)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:58.808420 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c43e038-9885-4ccd-af2a-19625f981c8b None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:58.811317 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 123/247] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 15 07:19:58 2022] GET /identity/v3/users/053b642391194fa899b9eca12ae3cf99 => generated 283 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:19:58.812283 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c43e038-9885-4ccd-af2a-19625f981c8b None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:58.863948 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49ae555-1741-4e65-ae4a-476db1f70b10 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:58.899025 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf9bce7-f2a3-4c2e-aa11-b83e72463e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:58.899636 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf9bce7-f2a3-4c2e-aa11-b83e72463e3a None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:58.899909 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf9bce7-f2a3-4c2e-aa11-b83e72463e3a None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:58.900174 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf9bce7-f2a3-4c2e-aa11-b83e72463e3a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:58.929791 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c43e038-9885-4ccd-af2a-19625f981c8b None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:58.965270 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:58.975555 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d513acc4-e123-43e5-a205-7baf27a5ad7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:58.976139 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf9bce7-f2a3-4c2e-aa11-b83e72463e3a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:58.976669 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d513acc4-e123-43e5-a205-7baf27a5ad7e None admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:58.979519 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d513acc4-e123-43e5-a205-7baf27a5ad7e None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:58.979519 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d513acc4-e123-43e5-a205-7baf27a5ad7e None admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:58.979672 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf9bce7-f2a3-4c2e-aa11-b83e72463e3a None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:58.979740 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d513acc4-e123-43e5-a205-7baf27a5ad7e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:58.980365 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d513acc4-e123-43e5-a205-7baf27a5ad7e None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:59.058562 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 124/248] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Sep 15 07:19:58 2022] POST /identity/v3/users => generated 293 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:19:59.059938 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d513acc4-e123-43e5-a205-7baf27a5ad7e None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:59.075432 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:59.094847 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 125/249] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 15 07:19:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 07:19:59.096577 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d2ceae-0b69-48e1-bd33-d869d145cd59 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:59.097044 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d2ceae-0b69-48e1-bd33-d869d145cd59 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:59.097492 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d2ceae-0b69-48e1-bd33-d869d145cd59 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:59.103307 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 126/250] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:19:59 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:19:59.104496 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d2ceae-0b69-48e1-bd33-d869d145cd59 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:59.197375 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d513acc4-e123-43e5-a205-7baf27a5ad7e None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:59.248923 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:59.261493 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d2ceae-0b69-48e1-bd33-d869d145cd59 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:59.262605 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c71934f-61fe-4fa9-a0a9-b5df4b04fe6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:59.262605 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c71934f-61fe-4fa9-a0a9-b5df4b04fe6d None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:59.262605 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c71934f-61fe-4fa9-a0a9-b5df4b04fe6d None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:59.262861 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c71934f-61fe-4fa9-a0a9-b5df4b04fe6d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:59.312647 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71e2951-2c3f-48e2-93a3-99aac5d83d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:59.313288 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71e2951-2c3f-48e2-93a3-99aac5d83d8a None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:59.316530 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71e2951-2c3f-48e2-93a3-99aac5d83d8a None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:59.316530 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71e2951-2c3f-48e2-93a3-99aac5d83d8a None admin] PATH_INFO: `/v3/projects/91757d436677474ab5f1027263238244` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:59.316530 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71e2951-2c3f-48e2-93a3-99aac5d83d8a None admin] RBAC: Authorizing `identity:get_project(project_id=91757d436677474ab5f1027263238244)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:59.317120 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71e2951-2c3f-48e2-93a3-99aac5d83d8a None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:59.323981 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 127/251] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 15 07:19:59 2022] GET /identity/v3/projects/91757d436677474ab5f1027263238244 => generated 302 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:19:59.325730 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d106db13-5413-42aa-baf7-64cb613898cd None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:59.325730 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d106db13-5413-42aa-baf7-64cb613898cd None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:59.325911 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d106db13-5413-42aa-baf7-64cb613898cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:59.353195 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-d106db13-5413-42aa-baf7-64cb613898cd None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:19:59.353195 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-d106db13-5413-42aa-baf7-64cb613898cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:19:59.359536 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c71934f-61fe-4fa9-a0a9-b5df4b04fe6d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:59.361040 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c71934f-61fe-4fa9-a0a9-b5df4b04fe6d None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:59.372088 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:59.445868 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 128/252] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:19:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 07:19:59.446938 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d106db13-5413-42aa-baf7-64cb613898cd None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:59.492547 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 125/253] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 15 07:19:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 07:19:59.505843 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c71934f-61fe-4fa9-a0a9-b5df4b04fe6d None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:59.569803 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:59.582917 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d106db13-5413-42aa-baf7-64cb613898cd None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:59.627445 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b88f36-54a4-4a83-a3f4-6fec4908c321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:59.628398 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b88f36-54a4-4a83-a3f4-6fec4908c321 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:59.628856 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b88f36-54a4-4a83-a3f4-6fec4908c321 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:59.629306 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b88f36-54a4-4a83-a3f4-6fec4908c321 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:59.630236 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b88f36-54a4-4a83-a3f4-6fec4908c321 None admin] RBAC: Adding query filter params (role.id=81b8a5256ba14184beb300947fbfe335, scope.project.id=91757d436677474ab5f1027263238244, user.id=44cdef2ad9f740ef8e5cbde0b851738e) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:19:59.630737 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b88f36-54a4-4a83-a3f4-6fec4908c321 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:59.632361 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b88f36-54a4-4a83-a3f4-6fec4908c321 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:59.639313 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 129/254] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 15 07:19:59 2022] GET /identity/v3/role_assignments?user.id=44cdef2ad9f740ef8e5cbde0b851738e&scope.project.id=91757d436677474ab5f1027263238244&role.id=81b8a5256ba14184beb300947fbfe335 => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:19:59.641116 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c71934f-61fe-4fa9-a0a9-b5df4b04fe6d None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:59.652192 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:19:59.680182 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39296a4-e98d-4a5e-9625-3cb33a966b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:59.681146 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39296a4-e98d-4a5e-9625-3cb33a966b40 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:59.681460 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39296a4-e98d-4a5e-9625-3cb33a966b40 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:59.681783 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39296a4-e98d-4a5e-9625-3cb33a966b40 None admin] PATH_INFO: `/v3/projects/91757d436677474ab5f1027263238244` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:59.684124 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39296a4-e98d-4a5e-9625-3cb33a966b40 None admin] RBAC: Authorizing `identity:get_project(project_id=91757d436677474ab5f1027263238244)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:59.686409 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39296a4-e98d-4a5e-9625-3cb33a966b40 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:59.693494 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 126/255] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 15 07:19:59 2022] GET /identity/v3/projects/91757d436677474ab5f1027263238244 => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:19:59.694879 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b88f36-54a4-4a83-a3f4-6fec4908c321 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:19:59.818704 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b88f36-54a4-4a83-a3f4-6fec4908c321 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:19:59.868677 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf7d415-dea9-4414-afa0-d71dbcfd0444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:19:59.869301 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf7d415-dea9-4414-afa0-d71dbcfd0444 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:19:59.869601 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf7d415-dea9-4414-afa0-d71dbcfd0444 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:19:59.869869 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf7d415-dea9-4414-afa0-d71dbcfd0444 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:19:59.870597 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf7d415-dea9-4414-afa0-d71dbcfd0444 None admin] RBAC: Adding query filter params (role.id=6bbb643e0d8d4bc0a9b27c26edf59216, scope.project.id=91757d436677474ab5f1027263238244, user.id=053b642391194fa899b9eca12ae3cf99) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:19:59.870905 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf7d415-dea9-4414-afa0-d71dbcfd0444 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:19:59.872373 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf7d415-dea9-4414-afa0-d71dbcfd0444 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:19:59.882639 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 130/256] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 15 07:19:59 2022] GET /identity/v3/role_assignments?user.id=053b642391194fa899b9eca12ae3cf99&scope.project.id=91757d436677474ab5f1027263238244&role.id=6bbb643e0d8d4bc0a9b27c26edf59216 => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:19:59.884755 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39296a4-e98d-4a5e-9625-3cb33a966b40 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:00.007557 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0282f659-460b-4a01-ac8b-0361edc3d580 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:00.007897 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0282f659-460b-4a01-ac8b-0361edc3d580 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:00.008104 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0282f659-460b-4a01-ac8b-0361edc3d580 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:00.009488 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 131/257] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:20:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 07:20:00.015604 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3869d725-065c-4e7b-965b-fc79d208f1ff None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:00.015885 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3869d725-065c-4e7b-965b-fc79d208f1ff None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:00.016165 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3869d725-065c-4e7b-965b-fc79d208f1ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:00.021211 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39296a4-e98d-4a5e-9625-3cb33a966b40 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:00.062128 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-3869d725-065c-4e7b-965b-fc79d208f1ff None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:20:00.063050 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e3e0a6-039a-4d34-bf36-7ab9da4bd0d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:00.063367 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-3869d725-065c-4e7b-965b-fc79d208f1ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:20:00.063775 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e3e0a6-039a-4d34-bf36-7ab9da4bd0d3 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:00.064064 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e3e0a6-039a-4d34-bf36-7ab9da4bd0d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:00.064343 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e3e0a6-039a-4d34-bf36-7ab9da4bd0d3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:00.077063 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e3e0a6-039a-4d34-bf36-7ab9da4bd0d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:20:00.082239 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-51e3e0a6-039a-4d34-bf36-7ab9da4bd0d3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 07:20:00.082239 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:00.082239 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:00.082239 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:00.082239 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:00.082239 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:00.082239 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:00.082239 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:00.082239 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:00.082239 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:00.082239 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:00.082239 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:00.082239 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:20:00.082239 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:20:00.082239 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:20:00.082239 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:20:00.083149 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:20:00.083149 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:20:00.083149 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:20:00.083149 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:20:00.083149 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:00.083149 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:00.083149 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:00.083149 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:00.083149 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:00.083149 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:00.083149 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:00.083149 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:00.083149 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:00.083149 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:00.083149 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:00.083149 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:00.083940 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:00.083940 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:00.083940 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:00.083940 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:00.083940 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:20:00.083940 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:20:00.083940 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:20:00.083940 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:20:00.083940 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:20:00.083940 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:20:00.083940 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 07:20:00.083940 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:20:00.083940 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 127/258] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 15 07:19:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:00.090832 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e3e0a6-039a-4d34-bf36-7ab9da4bd0d3 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:00.103637 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:00.135252 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:00.229605 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e3e0a6-039a-4d34-bf36-7ab9da4bd0d3 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:00.263128 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 132/259] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:20:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 07:20:00.272876 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215845ef-a5d5-4e17-8215-ad0284072671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:00.273803 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215845ef-a5d5-4e17-8215-ad0284072671 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:00.274291 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215845ef-a5d5-4e17-8215-ad0284072671 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:00.274765 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215845ef-a5d5-4e17-8215-ad0284072671 None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:00.275653 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215845ef-a5d5-4e17-8215-ad0284072671 None admin] RBAC: Adding query filter params (name=member) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:00.276222 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215845ef-a5d5-4e17-8215-ad0284072671 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:00.277832 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215845ef-a5d5-4e17-8215-ad0284072671 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:00.287371 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 128/260] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Sep 15 07:20:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:00.293129 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3869d725-065c-4e7b-965b-fc79d208f1ff None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:00.399812 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3869d725-065c-4e7b-965b-fc79d208f1ff None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:00.436756 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9216e8b2-f9e6-46e0-b902-a5f8b8fddf70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:00.437885 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9216e8b2-f9e6-46e0-b902-a5f8b8fddf70 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:00.438430 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9216e8b2-f9e6-46e0-b902-a5f8b8fddf70 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:00.438904 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9216e8b2-f9e6-46e0-b902-a5f8b8fddf70 None admin] PATH_INFO: `/v3/users/0339c9cb36724417a2680476b09e7033` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:00.440281 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9216e8b2-f9e6-46e0-b902-a5f8b8fddf70 None admin] RBAC: Authorizing `identity:get_user(user_id=0339c9cb36724417a2680476b09e7033)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:00.441979 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9216e8b2-f9e6-46e0-b902-a5f8b8fddf70 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:00.444072 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 133/261] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 15 07:20:00 2022] GET /identity/v3/users/0339c9cb36724417a2680476b09e7033 => generated 297 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:20:00.449382 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215845ef-a5d5-4e17-8215-ad0284072671 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:00.573207 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:00.585278 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215845ef-a5d5-4e17-8215-ad0284072671 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:00.631230 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeff2fc-c83a-43ff-9826-e6ea21675961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:00.632225 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeff2fc-c83a-43ff-9826-e6ea21675961 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:00.632708 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeff2fc-c83a-43ff-9826-e6ea21675961 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:00.633201 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeff2fc-c83a-43ff-9826-e6ea21675961 None admin] PATH_INFO: `/v3/projects/91757d436677474ab5f1027263238244` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:00.634586 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeff2fc-c83a-43ff-9826-e6ea21675961 None admin] RBAC: Authorizing `identity:get_project(project_id=91757d436677474ab5f1027263238244)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:00.636260 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeff2fc-c83a-43ff-9826-e6ea21675961 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:00.638166 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 129/262] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 15 07:20:00 2022] GET /identity/v3/projects/91757d436677474ab5f1027263238244 => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:00.644765 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9216e8b2-f9e6-46e0-b902-a5f8b8fddf70 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:00.681180 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:00.727935 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeff2fc-c83a-43ff-9826-e6ea21675961 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:00.805611 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9216e8b2-f9e6-46e0-b902-a5f8b8fddf70 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:00.847273 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962f1069-0d93-4cfa-848f-9933efd834b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:00.848882 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962f1069-0d93-4cfa-848f-9933efd834b9 None admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:00.849414 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962f1069-0d93-4cfa-848f-9933efd834b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:00.850014 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962f1069-0d93-4cfa-848f-9933efd834b9 None admin] PATH_INFO: `/v3/projects/91757d436677474ab5f1027263238244/users/0339c9cb36724417a2680476b09e7033/roles/053e12af84e943e699460a9d76c2072b` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:00.852239 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962f1069-0d93-4cfa-848f-9933efd834b9 None admin] RBAC: Authorizing `identity:create_grant(project_id=91757d436677474ab5f1027263238244, user_id=0339c9cb36724417a2680476b09e7033, role_id=053e12af84e943e699460a9d76c2072b)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:00.853973 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962f1069-0d93-4cfa-848f-9933efd834b9 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:00.873304 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeff2fc-c83a-43ff-9826-e6ea21675961 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:00.913782 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89ddc56-4d2f-4767-aff9-1d71bfdf0a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:00.914425 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89ddc56-4d2f-4767-aff9-1d71bfdf0a15 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:00.914706 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89ddc56-4d2f-4767-aff9-1d71bfdf0a15 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:00.915079 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89ddc56-4d2f-4767-aff9-1d71bfdf0a15 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:00.929443 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 134/263] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Sep 15 07:20:00 2022] PUT /identity/v3/projects/91757d436677474ab5f1027263238244/users/0339c9cb36724417a2680476b09e7033/roles/053e12af84e943e699460a9d76c2072b => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:20:00.933690 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89ddc56-4d2f-4767-aff9-1d71bfdf0a15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:20:00.936667 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-c89ddc56-4d2f-4767-aff9-1d71bfdf0a15 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 07:20:00.936667 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:00.936667 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:00.936667 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:00.936667 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:00.936667 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:00.936667 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:00.936667 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:00.936667 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:00.936667 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:00.936667 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:00.936667 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:00.936667 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:20:00.936667 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:20:00.936667 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:20:00.936667 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:20:00.937601 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:20:00.937601 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:20:00.937601 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:20:00.937601 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:20:00.937601 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:00.937601 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:00.937601 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:00.937601 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:00.937601 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:00.937601 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:00.937601 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:00.937601 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:00.937601 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:00.937601 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:00.937601 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:00.937601 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:00.938434 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:00.938434 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:00.938434 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:00.938434 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:00.938434 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:20:00.938434 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:20:00.938434 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:20:00.938434 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:20:00.938434 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:20:00.938434 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:20:00.938434 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 07:20:00.938434 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:20:00.938434 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 130/264] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Sep 15 07:20:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 07:20:00.944302 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962f1069-0d93-4cfa-848f-9933efd834b9 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:01.051937 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:01.085471 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962f1069-0d93-4cfa-848f-9933efd834b9 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:01.148552 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f885da9-776f-4628-b2c4-ac6377737ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:01.149506 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f885da9-776f-4628-b2c4-ac6377737ac2 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:01.149980 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f885da9-776f-4628-b2c4-ac6377737ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:01.150418 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f885da9-776f-4628-b2c4-ac6377737ac2 None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:01.151284 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f885da9-776f-4628-b2c4-ac6377737ac2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:01.151758 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f885da9-776f-4628-b2c4-ac6377737ac2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:01.153308 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f885da9-776f-4628-b2c4-ac6377737ac2 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:01.166499 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 135/265] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 15 07:20:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:01.173198 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89ddc56-4d2f-4767-aff9-1d71bfdf0a15 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:01.225745 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:01.315294 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89ddc56-4d2f-4767-aff9-1d71bfdf0a15 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:01.355487 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf56723d-493b-43d8-80ba-ccb58f0d1b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:01.356415 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf56723d-493b-43d8-80ba-ccb58f0d1b11 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:01.356869 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf56723d-493b-43d8-80ba-ccb58f0d1b11 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:01.357898 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf56723d-493b-43d8-80ba-ccb58f0d1b11 None admin] PATH_INFO: `/v3/users/053b642391194fa899b9eca12ae3cf99` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:01.359562 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf56723d-493b-43d8-80ba-ccb58f0d1b11 None admin] RBAC: Authorizing `identity:get_user(user_id=053b642391194fa899b9eca12ae3cf99)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:01.362466 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf56723d-493b-43d8-80ba-ccb58f0d1b11 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:01.365859 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 131/266] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 15 07:20:01 2022] GET /identity/v3/users/053b642391194fa899b9eca12ae3cf99 => generated 283 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:01.374435 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f885da9-776f-4628-b2c4-ac6377737ac2 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:01.492302 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f885da9-776f-4628-b2c4-ac6377737ac2 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:01.536288 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984a00b2-5236-4d0b-94be-d96ee16d7efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:01.537198 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984a00b2-5236-4d0b-94be-d96ee16d7efa None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:01.537694 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984a00b2-5236-4d0b-94be-d96ee16d7efa None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:01.538136 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984a00b2-5236-4d0b-94be-d96ee16d7efa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:01.550663 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:01.642619 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984a00b2-5236-4d0b-94be-d96ee16d7efa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:01.644669 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984a00b2-5236-4d0b-94be-d96ee16d7efa None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:01.752879 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 136/267] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 15 07:20:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 15 07:20:01.758702 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf56723d-493b-43d8-80ba-ccb58f0d1b11 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:01.858952 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:01.866420 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf56723d-493b-43d8-80ba-ccb58f0d1b11 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:01.902671 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45067f41-b287-410e-a15c-3491a77fe550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:01.903654 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45067f41-b287-410e-a15c-3491a77fe550 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:01.904319 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45067f41-b287-410e-a15c-3491a77fe550 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:01.904768 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45067f41-b287-410e-a15c-3491a77fe550 None admin] PATH_INFO: `/v3/projects/91757d436677474ab5f1027263238244` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:01.906147 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45067f41-b287-410e-a15c-3491a77fe550 None admin] RBAC: Authorizing `identity:get_project(project_id=91757d436677474ab5f1027263238244)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:01.907746 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45067f41-b287-410e-a15c-3491a77fe550 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:01.909565 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 132/268] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 15 07:20:01 2022] GET /identity/v3/projects/91757d436677474ab5f1027263238244 => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:01.918678 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984a00b2-5236-4d0b-94be-d96ee16d7efa None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:01.972964 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:02.067114 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984a00b2-5236-4d0b-94be-d96ee16d7efa None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:02.114607 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c1e017-9aa2-407f-9809-8161fe3f8da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:02.115225 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c1e017-9aa2-407f-9809-8161fe3f8da0 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:02.115504 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c1e017-9aa2-407f-9809-8161fe3f8da0 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:02.115785 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c1e017-9aa2-407f-9809-8161fe3f8da0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:02.116513 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c1e017-9aa2-407f-9809-8161fe3f8da0 None admin] RBAC: Adding query filter params (role.id=a0fae926490542b4977d1ecf27fcece4, scope.project.id=91757d436677474ab5f1027263238244, user.id=053b642391194fa899b9eca12ae3cf99) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:02.116822 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c1e017-9aa2-407f-9809-8161fe3f8da0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:02.118308 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c1e017-9aa2-407f-9809-8161fe3f8da0 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:02.124546 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 137/269] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 15 07:20:01 2022] GET /identity/v3/role_assignments?user.id=053b642391194fa899b9eca12ae3cf99&scope.project.id=91757d436677474ab5f1027263238244&role.id=a0fae926490542b4977d1ecf27fcece4 => generated 608 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:02.342527 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaaceb2-41fb-44a6-9f04-9709717feb8a None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:02.342715 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaaceb2-41fb-44a6-9f04-9709717feb8a None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:02.343146 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaaceb2-41fb-44a6-9f04-9709717feb8a None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:02.344867 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 133/270] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:20:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:20:02.350825 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbbd26a-2976-4ce5-a83e-958fe701300b None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:02.351117 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbbd26a-2976-4ce5-a83e-958fe701300b None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:02.351469 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbbd26a-2976-4ce5-a83e-958fe701300b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:02.373890 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-7fbbd26a-2976-4ce5-a83e-958fe701300b None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:20:02.375088 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-7fbbd26a-2976-4ce5-a83e-958fe701300b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:20:02.401405 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3adc1b9-5d00-40c7-a4be-304f93c1c688 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:02.401721 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3adc1b9-5d00-40c7-a4be-304f93c1c688 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:02.401992 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3adc1b9-5d00-40c7-a4be-304f93c1c688 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:02.403556 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 134/271] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:20:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:20:02.416596 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1159f1e4-9b28-48f7-96e2-13d4a0dd9e0f None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:02.416876 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1159f1e4-9b28-48f7-96e2-13d4a0dd9e0f None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:02.417160 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1159f1e4-9b28-48f7-96e2-13d4a0dd9e0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:02.447318 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 138/272] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:20:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 07:20:02.452129 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-1159f1e4-9b28-48f7-96e2-13d4a0dd9e0f None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:20:02.454934 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-1159f1e4-9b28-48f7-96e2-13d4a0dd9e0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:20:02.535718 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 135/273] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:20:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 07:20:02.585060 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1e47cc-3647-49a6-b5b3-9ef067b6954f None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:02.585392 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1e47cc-3647-49a6-b5b3-9ef067b6954f None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:02.585675 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1e47cc-3647-49a6-b5b3-9ef067b6954f None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:02.587314 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 139/274] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:20:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:20:02.592708 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f823b9-9f27-4bcb-b93e-2ee869700656 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:02.593229 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f823b9-9f27-4bcb-b93e-2ee869700656 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:02.593760 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f823b9-9f27-4bcb-b93e-2ee869700656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:02.613094 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-34f823b9-9f27-4bcb-b93e-2ee869700656 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:20:02.614637 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-34f823b9-9f27-4bcb-b93e-2ee869700656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:20:02.695354 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 136/275] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:20:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 07:20:02.740756 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1e47cc-3647-49a6-b5b3-9ef067b6954f None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:02.774396 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:02.831542 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f823b9-9f27-4bcb-b93e-2ee869700656 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:02.850002 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1e47cc-3647-49a6-b5b3-9ef067b6954f None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:02.858679 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:02.892369 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc9f89c-0849-44b8-adb6-c71ea3dfd2e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:02.893153 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc9f89c-0849-44b8-adb6-c71ea3dfd2e4 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:02.893588 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc9f89c-0849-44b8-adb6-c71ea3dfd2e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:02.893888 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc9f89c-0849-44b8-adb6-c71ea3dfd2e4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:02.895657 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc9f89c-0849-44b8-adb6-c71ea3dfd2e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:02.897692 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc9f89c-0849-44b8-adb6-c71ea3dfd2e4 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:02.900202 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 140/276] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:20:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:02.905334 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc9f89c-0849-44b8-adb6-c71ea3dfd2e4 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:02.945193 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f823b9-9f27-4bcb-b93e-2ee869700656 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:02.988898 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1c7ed6-fc82-4c4d-a169-dd7a697f2113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:02.989472 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c7ed6-fc82-4c4d-a169-dd7a697f2113 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:02.989860 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c7ed6-fc82-4c4d-a169-dd7a697f2113 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:02.990141 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c7ed6-fc82-4c4d-a169-dd7a697f2113 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:03.004212 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1c7ed6-fc82-4c4d-a169-dd7a697f2113 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:20:03.008751 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:03.009185 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-2e1c7ed6-fc82-4c4d-a169-dd7a697f2113 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 07:20:03.009185 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:03.009185 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:03.009185 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:03.009185 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:03.009185 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:03.009185 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:03.009185 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:03.009185 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:03.009185 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:03.009185 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:03.009185 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:03.009185 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:20:03.009185 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:20:03.009185 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:20:03.009185 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:20:03.010239 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:20:03.010239 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:20:03.010239 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:20:03.010239 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:20:03.010239 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:03.010239 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:03.010239 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:03.010239 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:03.010239 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:03.010239 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:03.010239 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:03.010239 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:03.010239 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:03.010239 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:03.010239 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:03.010239 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:03.011020 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:03.011020 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:03.011020 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:03.011020 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:03.011020 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:20:03.011020 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:20:03.011020 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:20:03.011020 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:20:03.011020 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:20:03.011020 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:20:03.011020 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 07:20:03.011020 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:20:03.011020 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 137/277] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 15 07:20:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:03.012224 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1c7ed6-fc82-4c4d-a169-dd7a697f2113 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:03.016369 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc9f89c-0849-44b8-adb6-c71ea3dfd2e4 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:03.053289 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981cd77e-e4c7-4702-8ac8-61124cd0a644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:03.053956 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981cd77e-e4c7-4702-8ac8-61124cd0a644 None admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:03.054236 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981cd77e-e4c7-4702-8ac8-61124cd0a644 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:03.054508 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981cd77e-e4c7-4702-8ac8-61124cd0a644 None admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:03.055161 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981cd77e-e4c7-4702-8ac8-61124cd0a644 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:03.056877 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981cd77e-e4c7-4702-8ac8-61124cd0a644 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:03.099660 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:03.117149 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1c7ed6-fc82-4c4d-a169-dd7a697f2113 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:03.130652 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 141/278] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Sep 15 07:20:02 2022] POST /identity/v3/users => generated 293 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 07:20:03.130652 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981cd77e-e4c7-4702-8ac8-61124cd0a644 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:03.169033 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8834c94d-c1b0-4e11-99ed-9e898ad7eb9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:03.170051 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8834c94d-c1b0-4e11-99ed-9e898ad7eb9d None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:03.170344 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8834c94d-c1b0-4e11-99ed-9e898ad7eb9d None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:03.170616 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8834c94d-c1b0-4e11-99ed-9e898ad7eb9d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:03.184213 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8834c94d-c1b0-4e11-99ed-9e898ad7eb9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:20:03.190240 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-8834c94d-c1b0-4e11-99ed-9e898ad7eb9d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 07:20:03.190240 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:03.190240 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:03.190240 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:03.190240 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:03.190240 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:03.190240 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:03.190240 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:03.190240 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:03.190240 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:03.190240 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:03.190240 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:03.190240 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:20:03.190240 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:20:03.190240 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:20:03.190240 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:20:03.191367 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:20:03.191367 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:20:03.191367 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:20:03.191367 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:20:03.191367 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:03.191367 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:03.191367 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:03.191367 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:03.191367 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:03.191367 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:03.191367 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:03.191367 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:03.191367 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:03.191367 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:03.191367 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:03.191367 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:03.192232 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:03.192232 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:03.192232 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:03.192232 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:03.192232 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:20:03.192232 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:20:03.192232 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:20:03.192232 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:20:03.192232 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:20:03.192232 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:20:03.192232 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 07:20:03.192232 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:20:03.192232 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 138/279] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 15 07:20:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:03.199205 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8834c94d-c1b0-4e11-99ed-9e898ad7eb9d None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:03.263933 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981cd77e-e4c7-4702-8ac8-61124cd0a644 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:03.309321 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d333e86e-85b7-4225-bfb9-56086ba4a1c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:03.311900 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d333e86e-85b7-4225-bfb9-56086ba4a1c5 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:03.312755 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d333e86e-85b7-4225-bfb9-56086ba4a1c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:03.313189 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d333e86e-85b7-4225-bfb9-56086ba4a1c5 None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:03.314352 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d333e86e-85b7-4225-bfb9-56086ba4a1c5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:03.314819 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d333e86e-85b7-4225-bfb9-56086ba4a1c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:03.317286 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d333e86e-85b7-4225-bfb9-56086ba4a1c5 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:03.325352 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8834c94d-c1b0-4e11-99ed-9e898ad7eb9d None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:03.331368 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 142/280] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Sep 15 07:20:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:03.340366 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d333e86e-85b7-4225-bfb9-56086ba4a1c5 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:03.365820 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:03.374291 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a6467a-40c1-43e0-975c-8beefb5c763c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:03.374847 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a6467a-40c1-43e0-975c-8beefb5c763c None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:03.375139 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a6467a-40c1-43e0-975c-8beefb5c763c None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:03.375416 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a6467a-40c1-43e0-975c-8beefb5c763c None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:03.376094 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a6467a-40c1-43e0-975c-8beefb5c763c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:03.376399 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a6467a-40c1-43e0-975c-8beefb5c763c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:03.377821 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a6467a-40c1-43e0-975c-8beefb5c763c None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:03.387562 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 139/281] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Sep 15 07:20:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:20:03.396132 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a6467a-40c1-43e0-975c-8beefb5c763c None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:03.401276 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:03.466298 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d333e86e-85b7-4225-bfb9-56086ba4a1c5 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:03.506803 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff04513-22bc-4343-8e58-4df833af44ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:03.507554 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff04513-22bc-4343-8e58-4df833af44ec None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:03.507834 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff04513-22bc-4343-8e58-4df833af44ec None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:03.508103 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff04513-22bc-4343-8e58-4df833af44ec None admin] PATH_INFO: `/v3/users/5913ef5cd6aa46dcb27a4ef829c420a8` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:03.530854 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff04513-22bc-4343-8e58-4df833af44ec None admin] RBAC: Authorizing `identity:get_user(user_id=5913ef5cd6aa46dcb27a4ef829c420a8)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:03.532476 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff04513-22bc-4343-8e58-4df833af44ec None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:03.534330 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 143/282] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 15 07:20:03 2022] GET /identity/v3/users/5913ef5cd6aa46dcb27a4ef829c420a8 => generated 293 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:20:03.540647 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff04513-22bc-4343-8e58-4df833af44ec None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:03.542936 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a6467a-40c1-43e0-975c-8beefb5c763c None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:03.594092 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4580062b-cf54-4eb4-86bf-936aa5c0557d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:03.594771 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4580062b-cf54-4eb4-86bf-936aa5c0557d None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:03.595062 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4580062b-cf54-4eb4-86bf-936aa5c0557d None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:03.595345 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4580062b-cf54-4eb4-86bf-936aa5c0557d None admin] PATH_INFO: `/v3/users/44cdef2ad9f740ef8e5cbde0b851738e` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:03.596610 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4580062b-cf54-4eb4-86bf-936aa5c0557d None admin] RBAC: Authorizing `identity:get_user(user_id=44cdef2ad9f740ef8e5cbde0b851738e)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:03.598183 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4580062b-cf54-4eb4-86bf-936aa5c0557d None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:03.600224 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 140/283] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 15 07:20:03 2022] GET /identity/v3/users/44cdef2ad9f740ef8e5cbde0b851738e => generated 297 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:20:03.606140 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4580062b-cf54-4eb4-86bf-936aa5c0557d None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:03.658749 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:03.687262 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff04513-22bc-4343-8e58-4df833af44ec None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:03.703105 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:03.726230 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8b9bbe-0cca-4fcc-a6d9-00ac6cdb1d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:03.726892 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8b9bbe-0cca-4fcc-a6d9-00ac6cdb1d5f None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:03.727178 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8b9bbe-0cca-4fcc-a6d9-00ac6cdb1d5f None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:03.727454 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8b9bbe-0cca-4fcc-a6d9-00ac6cdb1d5f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:03.728278 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8b9bbe-0cca-4fcc-a6d9-00ac6cdb1d5f None admin] RBAC: Adding query filter params (role.id=053e12af84e943e699460a9d76c2072b, user.id=5913ef5cd6aa46dcb27a4ef829c420a8, scope.system=all) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:03.728589 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8b9bbe-0cca-4fcc-a6d9-00ac6cdb1d5f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:03.730321 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8b9bbe-0cca-4fcc-a6d9-00ac6cdb1d5f None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:03.736903 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 144/284] 10.222.0.42 () {56 vars in 1308 bytes} [Thu Sep 15 07:20:03 2022] GET /identity/v3/role_assignments?user.id=5913ef5cd6aa46dcb27a4ef829c420a8&scope.system=all&role.id=053e12af84e943e699460a9d76c2072b => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:20:03.738422 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687c4a9f-c337-4cb6-b411-c02a46a97c96 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:03.738708 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687c4a9f-c337-4cb6-b411-c02a46a97c96 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:03.738998 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687c4a9f-c337-4cb6-b411-c02a46a97c96 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:03.740362 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 145/285] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:20:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:20:03.747169 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed1b984-9192-45dc-824e-d206cfef39ad None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:03.747480 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed1b984-9192-45dc-824e-d206cfef39ad None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:03.748515 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed1b984-9192-45dc-824e-d206cfef39ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:03.749105 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4580062b-cf54-4eb4-86bf-936aa5c0557d None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:03.772684 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-7ed1b984-9192-45dc-824e-d206cfef39ad None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:20:03.774974 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-7ed1b984-9192-45dc-824e-d206cfef39ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:20:03.796913 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b7e44b-6a7e-4f87-9e7c-c72ae0c2dd4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:03.798662 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b7e44b-6a7e-4f87-9e7c-c72ae0c2dd4f None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:03.798950 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b7e44b-6a7e-4f87-9e7c-c72ae0c2dd4f None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:03.799231 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b7e44b-6a7e-4f87-9e7c-c72ae0c2dd4f None admin] PATH_INFO: `/v3/projects/91757d436677474ab5f1027263238244` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:03.803117 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b7e44b-6a7e-4f87-9e7c-c72ae0c2dd4f None admin] RBAC: Authorizing `identity:get_project(project_id=91757d436677474ab5f1027263238244)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:03.804654 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b7e44b-6a7e-4f87-9e7c-c72ae0c2dd4f None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:03.813263 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 141/286] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 15 07:20:03 2022] GET /identity/v3/projects/91757d436677474ab5f1027263238244 => generated 302 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:20:03.821398 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b7e44b-6a7e-4f87-9e7c-c72ae0c2dd4f None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:03.910618 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 146/287] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:20:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 07:20:03.980391 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b7e44b-6a7e-4f87-9e7c-c72ae0c2dd4f None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:04.037059 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf6ab9f-5c5b-4e71-ba31-9a41fe20ebb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:04.037759 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf6ab9f-5c5b-4e71-ba31-9a41fe20ebb9 None admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:04.038043 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf6ab9f-5c5b-4e71-ba31-9a41fe20ebb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:04.038318 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf6ab9f-5c5b-4e71-ba31-9a41fe20ebb9 None admin] PATH_INFO: `/v3/projects/91757d436677474ab5f1027263238244/users/44cdef2ad9f740ef8e5cbde0b851738e/roles/81b8a5256ba14184beb300947fbfe335` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:04.040262 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf6ab9f-5c5b-4e71-ba31-9a41fe20ebb9 None admin] RBAC: Authorizing `identity:create_grant(project_id=91757d436677474ab5f1027263238244, user_id=44cdef2ad9f740ef8e5cbde0b851738e, role_id=81b8a5256ba14184beb300947fbfe335)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:04.041738 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf6ab9f-5c5b-4e71-ba31-9a41fe20ebb9 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:04.132481 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:04.136618 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 142/288] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Sep 15 07:20:03 2022] PUT /identity/v3/projects/91757d436677474ab5f1027263238244/users/44cdef2ad9f740ef8e5cbde0b851738e/roles/81b8a5256ba14184beb300947fbfe335 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:20:04.287589 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed1b984-9192-45dc-824e-d206cfef39ad None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:04.386545 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:04.425515 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed1b984-9192-45dc-824e-d206cfef39ad None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:04.464808 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547be10a-06bb-4334-aaeb-b9e525f7c636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:04.465464 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547be10a-06bb-4334-aaeb-b9e525f7c636 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:04.465775 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547be10a-06bb-4334-aaeb-b9e525f7c636 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:04.466054 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547be10a-06bb-4334-aaeb-b9e525f7c636 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:04.477601 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547be10a-06bb-4334-aaeb-b9e525f7c636 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:20:04.483479 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-547be10a-06bb-4334-aaeb-b9e525f7c636 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 07:20:04.483479 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:04.483479 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:04.483479 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:04.483479 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:04.483479 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:04.483479 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:04.483479 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:04.483479 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:04.483479 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:04.483479 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:04.483479 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:04.483479 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:20:04.483479 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:20:04.483479 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:20:04.483479 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:20:04.484901 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:20:04.484901 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:20:04.484901 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:20:04.484901 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:20:04.484901 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:04.484901 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:04.484901 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:04.484901 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:04.484901 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:04.484901 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:04.484901 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:04.484901 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:04.484901 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:04.484901 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:04.484901 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:04.484901 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:04.486052 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:04.486052 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:04.486052 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:04.486052 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:04.486052 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:20:04.486052 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:20:04.486052 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:20:04.486052 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:20:04.486052 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:20:04.486052 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:20:04.486052 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 07:20:04.486052 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:20:04.486052 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 147/289] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 15 07:20:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:04.490578 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf6ab9f-5c5b-4e71-ba31-9a41fe20ebb9 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:04.608012 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf6ab9f-5c5b-4e71-ba31-9a41fe20ebb9 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:04.663155 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729d7c92-ef38-4cf5-b8a5-30ceff17bfae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:04.663777 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729d7c92-ef38-4cf5-b8a5-30ceff17bfae None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:04.664066 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729d7c92-ef38-4cf5-b8a5-30ceff17bfae None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:04.664346 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729d7c92-ef38-4cf5-b8a5-30ceff17bfae None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:04.665022 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729d7c92-ef38-4cf5-b8a5-30ceff17bfae None admin] RBAC: Adding query filter params (name=member) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:04.665331 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729d7c92-ef38-4cf5-b8a5-30ceff17bfae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:04.666796 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729d7c92-ef38-4cf5-b8a5-30ceff17bfae None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:04.681686 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 143/290] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Sep 15 07:20:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:20:04.687709 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547be10a-06bb-4334-aaeb-b9e525f7c636 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:04.820453 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547be10a-06bb-4334-aaeb-b9e525f7c636 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:04.862096 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:04.871422 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd62d6f6-15d1-4f2c-beb5-1cafa24d50c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:04.872095 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd62d6f6-15d1-4f2c-beb5-1cafa24d50c4 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:04.877091 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd62d6f6-15d1-4f2c-beb5-1cafa24d50c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:04.877091 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd62d6f6-15d1-4f2c-beb5-1cafa24d50c4 None admin] PATH_INFO: `/v3/users/0339c9cb36724417a2680476b09e7033` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:04.878327 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd62d6f6-15d1-4f2c-beb5-1cafa24d50c4 None admin] RBAC: Authorizing `identity:get_user(user_id=0339c9cb36724417a2680476b09e7033)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:04.879795 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd62d6f6-15d1-4f2c-beb5-1cafa24d50c4 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:04.881390 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 148/291] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 15 07:20:04 2022] GET /identity/v3/users/0339c9cb36724417a2680476b09e7033 => generated 297 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:04.886946 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729d7c92-ef38-4cf5-b8a5-30ceff17bfae None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:04.972102 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:05.025522 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729d7c92-ef38-4cf5-b8a5-30ceff17bfae None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:05.078998 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d77ccad-a7ae-4135-9e5c-363c842450fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:05.079616 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d77ccad-a7ae-4135-9e5c-363c842450fb None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:05.079900 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d77ccad-a7ae-4135-9e5c-363c842450fb None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:05.080177 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d77ccad-a7ae-4135-9e5c-363c842450fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:05.166676 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d77ccad-a7ae-4135-9e5c-363c842450fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:05.168064 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d77ccad-a7ae-4135-9e5c-363c842450fb None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:05.246244 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:05.294551 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 144/292] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 15 07:20:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 07:20:05.299862 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd62d6f6-15d1-4f2c-beb5-1cafa24d50c4 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:05.428948 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd62d6f6-15d1-4f2c-beb5-1cafa24d50c4 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:05.478497 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323ad460-221f-4d34-8820-79104c66046a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:05.479148 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323ad460-221f-4d34-8820-79104c66046a None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:05.479430 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323ad460-221f-4d34-8820-79104c66046a None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:05.479707 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323ad460-221f-4d34-8820-79104c66046a None admin] PATH_INFO: `/v3/projects/91757d436677474ab5f1027263238244` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:05.486277 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323ad460-221f-4d34-8820-79104c66046a None admin] RBAC: Authorizing `identity:get_project(project_id=91757d436677474ab5f1027263238244)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:05.487809 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323ad460-221f-4d34-8820-79104c66046a None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:05.489472 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 149/293] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 15 07:20:05 2022] GET /identity/v3/projects/91757d436677474ab5f1027263238244 => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:05.495330 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d77ccad-a7ae-4135-9e5c-363c842450fb None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:05.616374 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d77ccad-a7ae-4135-9e5c-363c842450fb None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:05.651288 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237142a7-ee6c-4c87-84fb-2b309e0de9d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:05.651896 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237142a7-ee6c-4c87-84fb-2b309e0de9d1 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:05.652179 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237142a7-ee6c-4c87-84fb-2b309e0de9d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:05.652454 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237142a7-ee6c-4c87-84fb-2b309e0de9d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:05.653184 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237142a7-ee6c-4c87-84fb-2b309e0de9d1 None admin] RBAC: Adding query filter params (role.id=053e12af84e943e699460a9d76c2072b, scope.project.id=91757d436677474ab5f1027263238244, user.id=0339c9cb36724417a2680476b09e7033) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:05.653496 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237142a7-ee6c-4c87-84fb-2b309e0de9d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:05.656281 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237142a7-ee6c-4c87-84fb-2b309e0de9d1 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:05.666609 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 145/294] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 15 07:20:05 2022] GET /identity/v3/role_assignments?user.id=0339c9cb36724417a2680476b09e7033&scope.project.id=91757d436677474ab5f1027263238244&role.id=053e12af84e943e699460a9d76c2072b => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:06.268154 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1071e3-08d8-4265-a82b-f8442b284f55 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:06.268340 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1071e3-08d8-4265-a82b-f8442b284f55 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:06.268751 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1071e3-08d8-4265-a82b-f8442b284f55 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:06.270510 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 150/295] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:20:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:20:06.275551 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c4830-9c36-41bb-aaf7-10a7463f779a None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:06.275984 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c4830-9c36-41bb-aaf7-10a7463f779a None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:06.276157 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c4830-9c36-41bb-aaf7-10a7463f779a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:06.283329 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5840f3bf-6570-4184-ac0c-2f0b485ec599 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:06.284079 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5840f3bf-6570-4184-ac0c-2f0b485ec599 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:06.284578 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5840f3bf-6570-4184-ac0c-2f0b485ec599 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:06.287038 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 151/296] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:20:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 07:20:06.292422 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01a4e75-6e39-4fbd-8083-8937dd171b89 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:06.293440 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01a4e75-6e39-4fbd-8083-8937dd171b89 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:06.293950 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01a4e75-6e39-4fbd-8083-8937dd171b89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:06.299715 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-1c1c4830-9c36-41bb-aaf7-10a7463f779a None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:20:06.300909 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-1c1c4830-9c36-41bb-aaf7-10a7463f779a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:20:06.316628 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-d01a4e75-6e39-4fbd-8083-8937dd171b89 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:20:06.318498 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-d01a4e75-6e39-4fbd-8083-8937dd171b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:20:06.331916 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:06.393125 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 146/297] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:20:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 07:20:06.394448 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8727a2-a02d-4d59-9a4c-34f8fa7c1a8e None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:06.394732 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8727a2-a02d-4d59-9a4c-34f8fa7c1a8e None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:06.395005 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8727a2-a02d-4d59-9a4c-34f8fa7c1a8e None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:06.396237 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 147/298] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:20:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:20:06.402990 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ffe1d-6df0-4db1-a41a-d675a678e75a None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:06.404818 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ffe1d-6df0-4db1-a41a-d675a678e75a None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:06.405165 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ffe1d-6df0-4db1-a41a-d675a678e75a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:06.408167 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:06.414275 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 152/299] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:20:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 07:20:06.429523 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-089ffe1d-6df0-4db1-a41a-d675a678e75a None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:20:06.430784 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-089ffe1d-6df0-4db1-a41a-d675a678e75a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:20:06.500548 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 148/300] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:20:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 07:20:06.664624 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01a4e75-6e39-4fbd-8083-8937dd171b89 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:06.705414 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089ffe1d-6df0-4db1-a41a-d675a678e75a None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:06.784753 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01a4e75-6e39-4fbd-8083-8937dd171b89 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:06.799496 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:06.815505 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089ffe1d-6df0-4db1-a41a-d675a678e75a None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:06.825198 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3204e8-2c6c-41cb-bd81-cc608a70a17f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:06.826584 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3204e8-2c6c-41cb-bd81-cc608a70a17f None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:06.826914 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3204e8-2c6c-41cb-bd81-cc608a70a17f None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:06.827184 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3204e8-2c6c-41cb-bd81-cc608a70a17f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:06.838323 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3204e8-2c6c-41cb-bd81-cc608a70a17f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:20:06.843124 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-6b3204e8-2c6c-41cb-bd81-cc608a70a17f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 07:20:06.843124 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:06.843124 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:06.843124 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:06.843124 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:06.843124 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:06.843124 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:06.843124 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:06.843124 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:06.843124 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:06.843124 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:06.843124 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:06.843124 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:20:06.843124 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:20:06.843124 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:20:06.843124 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:20:06.843973 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:20:06.843973 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:20:06.843973 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:20:06.843973 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:20:06.843973 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:06.843973 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:06.843973 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:06.843973 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:06.843973 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:06.843973 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:06.843973 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:06.843973 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:06.843973 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:06.843973 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:06.843973 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:06.843973 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:06.849810 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:06.849810 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:06.849810 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:06.849810 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:06.849810 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:20:06.849810 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:20:06.849810 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:20:06.849810 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:20:06.849810 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:20:06.849810 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:20:06.849810 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 07:20:06.849810 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:20:06.849810 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 153/301] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 15 07:20:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:06.849810 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3204e8-2c6c-41cb-bd81-cc608a70a17f None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:06.854082 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19150499-b3ae-4920-9fc2-80f88371b7da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:06.854717 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19150499-b3ae-4920-9fc2-80f88371b7da None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:06.855071 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19150499-b3ae-4920-9fc2-80f88371b7da None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:06.855348 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19150499-b3ae-4920-9fc2-80f88371b7da None admin] PATH_INFO: `/v3/roles/member` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:06.865459 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19150499-b3ae-4920-9fc2-80f88371b7da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:20:06.870368 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-19150499-b3ae-4920-9fc2-80f88371b7da None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 07:20:06.870368 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:06.870368 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:06.870368 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:06.870368 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:06.870368 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:06.870368 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:06.870368 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:06.870368 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:06.870368 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:06.870368 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:06.870368 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:06.870368 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:20:06.870368 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:20:06.870368 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:20:06.870368 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:20:06.871240 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:20:06.871240 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:20:06.871240 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:20:06.871240 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:20:06.871240 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:06.871240 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:06.871240 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:06.871240 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:06.871240 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:06.871240 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:06.871240 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:06.871240 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:06.871240 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:06.871240 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:06.871240 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:06.871240 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:06.872038 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:06.872038 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:06.872038 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:06.872038 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:06.872038 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:20:06.872038 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:20:06.872038 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:20:06.872038 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:20:06.872038 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:20:06.872038 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:20:06.872038 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 07:20:06.872038 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:20:06.872038 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 149/302] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 15 07:20:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 07:20:06.872675 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19150499-b3ae-4920-9fc2-80f88371b7da None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:06.907660 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:06.953009 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3204e8-2c6c-41cb-bd81-cc608a70a17f None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:06.974381 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19150499-b3ae-4920-9fc2-80f88371b7da None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:06.989416 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a974c0ff-2691-471c-beed-0c55e8d3077d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:06.990085 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a974c0ff-2691-471c-beed-0c55e8d3077d None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:06.990388 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a974c0ff-2691-471c-beed-0c55e8d3077d None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:06.990657 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a974c0ff-2691-471c-beed-0c55e8d3077d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:06.991756 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a974c0ff-2691-471c-beed-0c55e8d3077d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:06.993185 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a974c0ff-2691-471c-beed-0c55e8d3077d None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:06.994787 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 154/303] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:20:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:06.995887 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c23bc4-906d-40f2-893f-077e5a4555e1 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:06.996166 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c23bc4-906d-40f2-893f-077e5a4555e1 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:06.996450 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c23bc4-906d-40f2-893f-077e5a4555e1 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:06.997647 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 155/304] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:20:06 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:20:06.998421 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c23bc4-906d-40f2-893f-077e5a4555e1 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:07.013215 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd69a49-4d1a-4544-b3b4-c35a53398bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:07.014329 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd69a49-4d1a-4544-b3b4-c35a53398bf6 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:07.014617 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd69a49-4d1a-4544-b3b4-c35a53398bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:07.014897 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd69a49-4d1a-4544-b3b4-c35a53398bf6 None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:07.015565 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd69a49-4d1a-4544-b3b4-c35a53398bf6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:07.015872 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd69a49-4d1a-4544-b3b4-c35a53398bf6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:07.017233 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd69a49-4d1a-4544-b3b4-c35a53398bf6 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:07.025226 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 150/305] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Sep 15 07:20:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:07.026105 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd69a49-4d1a-4544-b3b4-c35a53398bf6 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:07.037859 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:07.104319 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c23bc4-906d-40f2-893f-077e5a4555e1 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:07.127128 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd69a49-4d1a-4544-b3b4-c35a53398bf6 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:07.127644 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:07.137902 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbccc2de-29d5-47a9-b4aa-15f9597b28a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:07.139052 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbccc2de-29d5-47a9-b4aa-15f9597b28a3 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:07.139381 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbccc2de-29d5-47a9-b4aa-15f9597b28a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:07.139654 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbccc2de-29d5-47a9-b4aa-15f9597b28a3 None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:07.140313 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbccc2de-29d5-47a9-b4aa-15f9597b28a3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:07.140618 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbccc2de-29d5-47a9-b4aa-15f9597b28a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:07.142018 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbccc2de-29d5-47a9-b4aa-15f9597b28a3 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:07.150375 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 156/306] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Sep 15 07:20:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:07.150375 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c740b37-8892-4ecb-9d2b-e0d38a51bae5 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:07.150375 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c740b37-8892-4ecb-9d2b-e0d38a51bae5 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:07.150375 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c740b37-8892-4ecb-9d2b-e0d38a51bae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:07.166391 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2803cd-2cc6-4818-866d-ffcf39cca3fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:07.167007 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2803cd-2cc6-4818-866d-ffcf39cca3fc None admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:07.167293 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2803cd-2cc6-4818-866d-ffcf39cca3fc None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:07.167568 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2803cd-2cc6-4818-866d-ffcf39cca3fc None admin] PATH_INFO: `/v3/groups` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:07.168204 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2803cd-2cc6-4818-866d-ffcf39cca3fc None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:07.169779 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2803cd-2cc6-4818-866d-ffcf39cca3fc None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:07.170482 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-0c740b37-8892-4ecb-9d2b-e0d38a51bae5 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:20:07.171671 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-0c740b37-8892-4ecb-9d2b-e0d38a51bae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:20:07.215826 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 151/307] 10.222.0.42 () {60 vars in 1134 bytes} [Thu Sep 15 07:20:07 2022] POST /identity/v3/groups => generated 229 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:20:07.216641 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2803cd-2cc6-4818-866d-ffcf39cca3fc None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:07.251257 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 157/308] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:20:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 07:20:07.251961 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c740b37-8892-4ecb-9d2b-e0d38a51bae5 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:07.323737 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:07.367606 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2803cd-2cc6-4818-866d-ffcf39cca3fc None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:07.387555 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c740b37-8892-4ecb-9d2b-e0d38a51bae5 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:07.410163 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:07.410457 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82eb9bd-7582-4122-ac56-14844e34733f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:07.411783 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82eb9bd-7582-4122-ac56-14844e34733f None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:07.411783 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82eb9bd-7582-4122-ac56-14844e34733f None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:07.411783 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82eb9bd-7582-4122-ac56-14844e34733f None admin] PATH_INFO: `/v3/users/7e9f8ae5dc9e4704860e97eacca341a7` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:07.435300 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2abf3f8-eca5-471b-bf0f-a64a2dece9d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:07.436016 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2abf3f8-eca5-471b-bf0f-a64a2dece9d1 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:07.436290 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2abf3f8-eca5-471b-bf0f-a64a2dece9d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:07.436564 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2abf3f8-eca5-471b-bf0f-a64a2dece9d1 None admin] PATH_INFO: `/v3/users/5913ef5cd6aa46dcb27a4ef829c420a8` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:07.436776 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82eb9bd-7582-4122-ac56-14844e34733f None admin] RBAC: Authorizing `identity:get_user(user_id=7e9f8ae5dc9e4704860e97eacca341a7)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:07.437727 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2abf3f8-eca5-471b-bf0f-a64a2dece9d1 None admin] RBAC: Authorizing `identity:get_user(user_id=5913ef5cd6aa46dcb27a4ef829c420a8)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:07.438623 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82eb9bd-7582-4122-ac56-14844e34733f None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:07.439151 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2abf3f8-eca5-471b-bf0f-a64a2dece9d1 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:07.440623 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 152/309] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 15 07:20:07 2022] GET /identity/v3/users/7e9f8ae5dc9e4704860e97eacca341a7 => generated 293 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:07.440734 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 158/310] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 15 07:20:07 2022] GET /identity/v3/users/5913ef5cd6aa46dcb27a4ef829c420a8 => generated 293 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:07.446487 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82eb9bd-7582-4122-ac56-14844e34733f None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:07.446621 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2abf3f8-eca5-471b-bf0f-a64a2dece9d1 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:07.571445 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82eb9bd-7582-4122-ac56-14844e34733f None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:07.633276 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2abf3f8-eca5-471b-bf0f-a64a2dece9d1 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:07.633452 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:07.633452 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19da1dc-66f9-4d02-ba40-db6f871432ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:07.633452 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19da1dc-66f9-4d02-ba40-db6f871432ca None admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:07.633452 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19da1dc-66f9-4d02-ba40-db6f871432ca None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:07.633790 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19da1dc-66f9-4d02-ba40-db6f871432ca None admin] PATH_INFO: `/v3/system/users/5913ef5cd6aa46dcb27a4ef829c420a8/roles/053e12af84e943e699460a9d76c2072b` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:07.633790 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19da1dc-66f9-4d02-ba40-db6f871432ca None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=5913ef5cd6aa46dcb27a4ef829c420a8, role_id=053e12af84e943e699460a9d76c2072b)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:07.633790 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19da1dc-66f9-4d02-ba40-db6f871432ca None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:07.634003 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b0444a-d9f3-4133-af1e-e0cf677b7be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:07.634003 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b0444a-d9f3-4133-af1e-e0cf677b7be1 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:07.634003 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b0444a-d9f3-4133-af1e-e0cf677b7be1 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:07.634003 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b0444a-d9f3-4133-af1e-e0cf677b7be1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:07.634276 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b0444a-d9f3-4133-af1e-e0cf677b7be1 None admin] RBAC: Adding query filter params (role.id=81b8a5256ba14184beb300947fbfe335, user.id=7e9f8ae5dc9e4704860e97eacca341a7, scope.system=all) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:07.634276 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b0444a-d9f3-4133-af1e-e0cf677b7be1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:07.634276 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b0444a-d9f3-4133-af1e-e0cf677b7be1 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:07.634276 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 159/311] 10.222.0.42 () {56 vars in 1308 bytes} [Thu Sep 15 07:20:07 2022] GET /identity/v3/role_assignments?user.id=7e9f8ae5dc9e4704860e97eacca341a7&scope.system=all&role.id=81b8a5256ba14184beb300947fbfe335 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:07.634276 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b0444a-d9f3-4133-af1e-e0cf677b7be1 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:07.674494 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 153/312] 10.222.0.42 () {58 vars in 1334 bytes} [Thu Sep 15 07:20:07 2022] PUT /identity/v3/system/users/5913ef5cd6aa46dcb27a4ef829c420a8/roles/053e12af84e943e699460a9d76c2072b => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:20:07.697962 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:07.768656 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b0444a-d9f3-4133-af1e-e0cf677b7be1 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:07.811875 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a7cf06-09c8-487a-8097-fb54e418813d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:07.812836 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a7cf06-09c8-487a-8097-fb54e418813d None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:07.813271 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a7cf06-09c8-487a-8097-fb54e418813d None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:07.813732 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a7cf06-09c8-487a-8097-fb54e418813d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:07.827764 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a7cf06-09c8-487a-8097-fb54e418813d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:20:07.833091 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-c8a7cf06-09c8-487a-8097-fb54e418813d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 07:20:07.833091 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:07.833091 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:07.833091 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:07.833091 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:07.833091 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:07.833091 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:07.833091 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:07.833091 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:07.833091 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:07.833091 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:07.833091 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:07.833091 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:20:07.833091 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:20:07.833091 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:20:07.833091 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:20:07.834318 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:20:07.834318 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:20:07.834318 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:20:07.834318 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:20:07.834318 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:07.834318 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:07.834318 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:07.834318 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:07.834318 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:07.834318 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:07.834318 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:07.834318 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:07.834318 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:07.834318 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:07.834318 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:07.834318 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:07.835334 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:07.835334 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:07.835334 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:07.835334 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:07.835334 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:20:07.835334 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:20:07.835334 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:20:07.835334 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:20:07.835334 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:20:07.835334 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:20:07.835334 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 07:20:07.835334 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:20:07.837417 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 160/313] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 15 07:20:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:07.843990 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19da1dc-66f9-4d02-ba40-db6f871432ca None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:07.953767 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19da1dc-66f9-4d02-ba40-db6f871432ca None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:07.990685 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700c562a-b80c-4fe1-a5cf-8b1ee682f99f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:07.990685 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700c562a-b80c-4fe1-a5cf-8b1ee682f99f None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:07.991005 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700c562a-b80c-4fe1-a5cf-8b1ee682f99f None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:07.991229 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700c562a-b80c-4fe1-a5cf-8b1ee682f99f None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:07.991902 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700c562a-b80c-4fe1-a5cf-8b1ee682f99f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:07.992203 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700c562a-b80c-4fe1-a5cf-8b1ee682f99f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:07.993571 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700c562a-b80c-4fe1-a5cf-8b1ee682f99f None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:08.004640 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 154/314] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Sep 15 07:20:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:08.010551 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a7cf06-09c8-487a-8097-fb54e418813d None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:08.117690 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a7cf06-09c8-487a-8097-fb54e418813d None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:08.127757 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:08.153028 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1563018-52e3-4f92-8d20-d3d1ceeaec84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:08.156388 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1563018-52e3-4f92-8d20-d3d1ceeaec84 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:08.156388 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1563018-52e3-4f92-8d20-d3d1ceeaec84 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:08.156388 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1563018-52e3-4f92-8d20-d3d1ceeaec84 None admin] PATH_INFO: `/v3/users/44cdef2ad9f740ef8e5cbde0b851738e` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:08.156388 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1563018-52e3-4f92-8d20-d3d1ceeaec84 None admin] RBAC: Authorizing `identity:get_user(user_id=44cdef2ad9f740ef8e5cbde0b851738e)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:08.157071 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1563018-52e3-4f92-8d20-d3d1ceeaec84 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:08.159037 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 161/315] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 15 07:20:08 2022] GET /identity/v3/users/44cdef2ad9f740ef8e5cbde0b851738e => generated 297 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:08.164659 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700c562a-b80c-4fe1-a5cf-8b1ee682f99f None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:08.223092 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:08.274098 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700c562a-b80c-4fe1-a5cf-8b1ee682f99f None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:08.309255 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b8436c-65a3-4d88-b08f-b5f4d8e8ee65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:08.309901 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b8436c-65a3-4d88-b08f-b5f4d8e8ee65 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:08.310180 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b8436c-65a3-4d88-b08f-b5f4d8e8ee65 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:08.310983 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b8436c-65a3-4d88-b08f-b5f4d8e8ee65 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:08.383485 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b8436c-65a3-4d88-b08f-b5f4d8e8ee65 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:08.384857 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b8436c-65a3-4d88-b08f-b5f4d8e8ee65 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:08.445977 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:08.489162 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 155/316] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 15 07:20:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 07:20:08.494730 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1563018-52e3-4f92-8d20-d3d1ceeaec84 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:08.601500 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1563018-52e3-4f92-8d20-d3d1ceeaec84 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:08.653169 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9634d1-a619-4118-8d68-e5ec92750287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:08.653763 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9634d1-a619-4118-8d68-e5ec92750287 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:08.654044 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9634d1-a619-4118-8d68-e5ec92750287 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:08.655000 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9634d1-a619-4118-8d68-e5ec92750287 None admin] PATH_INFO: `/v3/projects/91757d436677474ab5f1027263238244` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:08.656219 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9634d1-a619-4118-8d68-e5ec92750287 None admin] RBAC: Authorizing `identity:get_project(project_id=91757d436677474ab5f1027263238244)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:08.657716 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9634d1-a619-4118-8d68-e5ec92750287 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:08.659747 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 162/317] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 15 07:20:08 2022] GET /identity/v3/projects/91757d436677474ab5f1027263238244 => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:08.665457 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b8436c-65a3-4d88-b08f-b5f4d8e8ee65 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:08.766321 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b8436c-65a3-4d88-b08f-b5f4d8e8ee65 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:08.802814 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f49382b-bb9d-4ba4-9701-c3df7022da23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:08.803746 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f49382b-bb9d-4ba4-9701-c3df7022da23 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:08.804221 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f49382b-bb9d-4ba4-9701-c3df7022da23 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:08.804684 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f49382b-bb9d-4ba4-9701-c3df7022da23 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:08.805647 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f49382b-bb9d-4ba4-9701-c3df7022da23 None admin] RBAC: Adding query filter params (role.id=81b8a5256ba14184beb300947fbfe335, scope.project.id=91757d436677474ab5f1027263238244, user.id=44cdef2ad9f740ef8e5cbde0b851738e) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:08.806167 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f49382b-bb9d-4ba4-9701-c3df7022da23 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:08.808046 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f49382b-bb9d-4ba4-9701-c3df7022da23 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:08.814612 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 156/318] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 15 07:20:08 2022] GET /identity/v3/role_assignments?user.id=44cdef2ad9f740ef8e5cbde0b851738e&scope.project.id=91757d436677474ab5f1027263238244&role.id=81b8a5256ba14184beb300947fbfe335 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:09.628784 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaacb256-1c49-43c5-a219-66ea4bb4b0ad None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:09.629381 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaacb256-1c49-43c5-a219-66ea4bb4b0ad None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:09.629381 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaacb256-1c49-43c5-a219-66ea4bb4b0ad None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:09.631025 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 163/319] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:20:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:20:09.638371 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6992d712-6397-4e71-988d-ad6b973ec27a None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:09.639421 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6992d712-6397-4e71-988d-ad6b973ec27a None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:09.639545 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6992d712-6397-4e71-988d-ad6b973ec27a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:09.661664 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-6992d712-6397-4e71-988d-ad6b973ec27a None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:20:09.662886 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-6992d712-6397-4e71-988d-ad6b973ec27a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:20:09.679118 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:09.747522 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 157/320] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:20:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 07:20:09.974945 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2efcb8-f5ae-4d75-a462-9d1f7e68415e None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:09.976328 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2efcb8-f5ae-4d75-a462-9d1f7e68415e None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:09.978081 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2efcb8-f5ae-4d75-a462-9d1f7e68415e None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:09.980307 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 164/321] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:20:09 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:20:09.985651 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fde00a-c7e6-4d4a-b5ed-a1c3cc8cca1b None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:09.985999 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fde00a-c7e6-4d4a-b5ed-a1c3cc8cca1b None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:09.986207 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fde00a-c7e6-4d4a-b5ed-a1c3cc8cca1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:10.006797 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-37fde00a-c7e6-4d4a-b5ed-a1c3cc8cca1b None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:20:10.008020 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-37fde00a-c7e6-4d4a-b5ed-a1c3cc8cca1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:20:10.022548 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d90a067-39bc-4322-bbcc-c8c54672d58a None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:10.022881 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d90a067-39bc-4322-bbcc-c8c54672d58a None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:10.023114 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d90a067-39bc-4322-bbcc-c8c54672d58a None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:10.024759 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 165/322] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:20:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:20:10.028300 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d90a067-39bc-4322-bbcc-c8c54672d58a None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:10.083496 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:10.089267 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 158/323] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:20:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 07:20:10.090586 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e553a8e6-6266-46cf-827a-df8a4838a89a None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:10.090876 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e553a8e6-6266-46cf-827a-df8a4838a89a None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:10.091155 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e553a8e6-6266-46cf-827a-df8a4838a89a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:10.111030 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-e553a8e6-6266-46cf-827a-df8a4838a89a None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:20:10.112782 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-e553a8e6-6266-46cf-827a-df8a4838a89a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:20:10.149977 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d90a067-39bc-4322-bbcc-c8c54672d58a None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:10.195421 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c425fa-1157-4759-8260-3ca137fd5d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:10.196174 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c425fa-1157-4759-8260-3ca137fd5d36 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:10.196453 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c425fa-1157-4759-8260-3ca137fd5d36 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:10.196728 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c425fa-1157-4759-8260-3ca137fd5d36 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:10.198049 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c425fa-1157-4759-8260-3ca137fd5d36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:10.203214 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c425fa-1157-4759-8260-3ca137fd5d36 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:10.203214 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 166/324] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:20:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:10.208315 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c425fa-1157-4759-8260-3ca137fd5d36 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:10.213297 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 159/325] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:20:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 07:20:10.323447 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c425fa-1157-4759-8260-3ca137fd5d36 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:10.336937 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:10.358543 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38df4eb-d343-472d-8d5e-e1ff9f73af84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:10.359143 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38df4eb-d343-472d-8d5e-e1ff9f73af84 None admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:10.359416 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38df4eb-d343-472d-8d5e-e1ff9f73af84 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:10.359681 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38df4eb-d343-472d-8d5e-e1ff9f73af84 None admin] PATH_INFO: `/v3/groups` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:10.360310 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38df4eb-d343-472d-8d5e-e1ff9f73af84 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:10.361758 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38df4eb-d343-472d-8d5e-e1ff9f73af84 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:10.396831 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e553a8e6-6266-46cf-827a-df8a4838a89a None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:10.406149 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:10.457739 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 167/326] 10.222.0.42 () {60 vars in 1134 bytes} [Thu Sep 15 07:20:10 2022] POST /identity/v3/groups => generated 226 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 07:20:10.489848 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38df4eb-d343-472d-8d5e-e1ff9f73af84 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:10.503474 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e553a8e6-6266-46cf-827a-df8a4838a89a None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:10.543353 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9ed535-4ab5-4344-8b71-b484fc518d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:10.544191 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9ed535-4ab5-4344-8b71-b484fc518d8d None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:10.544477 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9ed535-4ab5-4344-8b71-b484fc518d8d None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:10.544794 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9ed535-4ab5-4344-8b71-b484fc518d8d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:10.557960 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9ed535-4ab5-4344-8b71-b484fc518d8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:20:10.563574 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-9d9ed535-4ab5-4344-8b71-b484fc518d8d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 07:20:10.563574 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:10.563574 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:10.563574 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:10.563574 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:10.563574 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:10.563574 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:10.563574 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:10.563574 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:10.563574 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:10.563574 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:10.563574 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:10.563574 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:20:10.563574 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:20:10.563574 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:20:10.563574 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:20:10.564535 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:20:10.564535 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:20:10.564535 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:20:10.564535 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:20:10.564535 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:10.564535 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:10.564535 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:10.564535 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:10.564535 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:10.564535 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:10.564535 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:10.564535 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:10.564535 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:10.564535 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:10.564535 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:10.564535 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:10.565304 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:10.565304 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:10.565304 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:10.565304 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:10.565304 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:20:10.565304 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:20:10.565304 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:20:10.565304 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:20:10.565304 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:20:10.565304 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:20:10.565304 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 07:20:10.565304 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:20:10.565304 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 160/327] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 15 07:20:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 07:20:10.570720 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9ed535-4ab5-4344-8b71-b484fc518d8d None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:10.609551 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38df4eb-d343-472d-8d5e-e1ff9f73af84 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:10.648555 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:10.649880 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9513807-101a-454e-97b9-feffe1ab8d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:10.651590 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9513807-101a-454e-97b9-feffe1ab8d85 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:10.651874 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9513807-101a-454e-97b9-feffe1ab8d85 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:10.652149 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9513807-101a-454e-97b9-feffe1ab8d85 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:10.664540 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9513807-101a-454e-97b9-feffe1ab8d85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:20:10.670173 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-f9513807-101a-454e-97b9-feffe1ab8d85 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 07:20:10.670173 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:10.670173 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:10.670173 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:10.670173 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:10.670173 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:10.670173 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:10.670173 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:10.670173 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:10.670173 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:10.670173 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:10.670173 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:10.670173 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:20:10.670173 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:20:10.670173 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:20:10.670173 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:20:10.671161 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:20:10.671161 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:20:10.671161 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:20:10.671161 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:20:10.671161 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:10.671161 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:10.671161 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:10.671161 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:10.671161 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:10.671161 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:10.671161 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:10.671161 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:10.671161 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:10.671161 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:10.671161 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:10.671161 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:10.672132 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:10.672132 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:10.672132 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:10.672132 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:10.672132 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:20:10.672132 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:20:10.672132 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:20:10.672132 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:20:10.672132 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:20:10.672132 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:20:10.672132 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 07:20:10.672132 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:20:10.672132 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 168/328] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 15 07:20:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 07:20:10.677630 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9513807-101a-454e-97b9-feffe1ab8d85 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:10.693557 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9ed535-4ab5-4344-8b71-b484fc518d8d None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:10.714407 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:10.743841 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2401f92-18e5-40e9-8c10-eaf1812d010c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:10.743841 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2401f92-18e5-40e9-8c10-eaf1812d010c None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:10.743841 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2401f92-18e5-40e9-8c10-eaf1812d010c None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:10.743841 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2401f92-18e5-40e9-8c10-eaf1812d010c None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:10.744270 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2401f92-18e5-40e9-8c10-eaf1812d010c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:10.744545 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2401f92-18e5-40e9-8c10-eaf1812d010c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:10.746972 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2401f92-18e5-40e9-8c10-eaf1812d010c None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:10.757965 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 161/329] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Sep 15 07:20:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:20:10.764506 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2401f92-18e5-40e9-8c10-eaf1812d010c None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:10.811641 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9513807-101a-454e-97b9-feffe1ab8d85 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:10.856463 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb62d521-ed5f-41eb-834c-a7ce97522ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:10.857120 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb62d521-ed5f-41eb-834c-a7ce97522ad5 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:10.858034 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb62d521-ed5f-41eb-834c-a7ce97522ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:10.858307 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb62d521-ed5f-41eb-834c-a7ce97522ad5 None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:10.858980 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb62d521-ed5f-41eb-834c-a7ce97522ad5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:10.859332 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb62d521-ed5f-41eb-834c-a7ce97522ad5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:10.860755 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb62d521-ed5f-41eb-834c-a7ce97522ad5 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:10.870927 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 169/330] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Sep 15 07:20:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:20:10.876709 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb62d521-ed5f-41eb-834c-a7ce97522ad5 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:10.894797 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2401f92-18e5-40e9-8c10-eaf1812d010c None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:10.928907 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:10.943764 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827f62c5-3574-40ae-81c5-f0dc6a6fc37f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:10.944713 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827f62c5-3574-40ae-81c5-f0dc6a6fc37f None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:10.945469 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827f62c5-3574-40ae-81c5-f0dc6a6fc37f None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:10.946435 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827f62c5-3574-40ae-81c5-f0dc6a6fc37f None admin] PATH_INFO: `/v3/users/7e9f8ae5dc9e4704860e97eacca341a7` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:10.952835 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827f62c5-3574-40ae-81c5-f0dc6a6fc37f None admin] RBAC: Authorizing `identity:get_user(user_id=7e9f8ae5dc9e4704860e97eacca341a7)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:10.954764 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827f62c5-3574-40ae-81c5-f0dc6a6fc37f None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:10.958323 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 162/331] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 15 07:20:10 2022] GET /identity/v3/users/7e9f8ae5dc9e4704860e97eacca341a7 => generated 293 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:20:10.963776 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827f62c5-3574-40ae-81c5-f0dc6a6fc37f None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:10.998183 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:11.007587 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb62d521-ed5f-41eb-834c-a7ce97522ad5 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:11.052908 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e333c3d0-5673-415b-a73f-4667393eebb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:11.053284 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e333c3d0-5673-415b-a73f-4667393eebb6 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:11.053611 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e333c3d0-5673-415b-a73f-4667393eebb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:11.053900 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e333c3d0-5673-415b-a73f-4667393eebb6 None admin] PATH_INFO: `/v3/users/5913ef5cd6aa46dcb27a4ef829c420a8` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:11.055381 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e333c3d0-5673-415b-a73f-4667393eebb6 None admin] RBAC: Authorizing `identity:get_user(user_id=5913ef5cd6aa46dcb27a4ef829c420a8)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:11.057281 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e333c3d0-5673-415b-a73f-4667393eebb6 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:11.059855 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 170/332] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 15 07:20:10 2022] GET /identity/v3/users/5913ef5cd6aa46dcb27a4ef829c420a8 => generated 293 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:20:11.065862 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e333c3d0-5673-415b-a73f-4667393eebb6 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:11.116885 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827f62c5-3574-40ae-81c5-f0dc6a6fc37f None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:11.160149 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db4cbbf-be5d-4e4c-850d-f518fa45d521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:11.161326 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db4cbbf-be5d-4e4c-850d-f518fa45d521 None admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:11.161879 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db4cbbf-be5d-4e4c-850d-f518fa45d521 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:11.162424 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db4cbbf-be5d-4e4c-850d-f518fa45d521 None admin] PATH_INFO: `/v3/system/users/7e9f8ae5dc9e4704860e97eacca341a7/roles/81b8a5256ba14184beb300947fbfe335` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:11.164194 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db4cbbf-be5d-4e4c-850d-f518fa45d521 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=7e9f8ae5dc9e4704860e97eacca341a7, role_id=81b8a5256ba14184beb300947fbfe335)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:11.165981 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db4cbbf-be5d-4e4c-850d-f518fa45d521 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:11.195115 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e333c3d0-5673-415b-a73f-4667393eebb6 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:11.229280 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 163/333] 10.222.0.42 () {58 vars in 1334 bytes} [Thu Sep 15 07:20:10 2022] PUT /identity/v3/system/users/7e9f8ae5dc9e4704860e97eacca341a7/roles/81b8a5256ba14184beb300947fbfe335 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:20:11.234489 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb31053-2152-426f-a168-c5578aef7a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:11.235516 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb31053-2152-426f-a168-c5578aef7a67 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:11.241103 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb31053-2152-426f-a168-c5578aef7a67 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:11.241103 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb31053-2152-426f-a168-c5578aef7a67 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:11.241103 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb31053-2152-426f-a168-c5578aef7a67 None admin] RBAC: Adding query filter params (role.id=053e12af84e943e699460a9d76c2072b, user.id=5913ef5cd6aa46dcb27a4ef829c420a8, scope.system=all) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:11.241103 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb31053-2152-426f-a168-c5578aef7a67 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:11.241103 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb31053-2152-426f-a168-c5578aef7a67 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:11.259738 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 171/334] 10.222.0.42 () {56 vars in 1308 bytes} [Thu Sep 15 07:20:11 2022] GET /identity/v3/role_assignments?user.id=5913ef5cd6aa46dcb27a4ef829c420a8&scope.system=all&role.id=053e12af84e943e699460a9d76c2072b => generated 510 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:13.497492 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d4ad90-7c20-4a1a-a409-b974c817f080 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:13.498556 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d4ad90-7c20-4a1a-a409-b974c817f080 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:13.498639 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d4ad90-7c20-4a1a-a409-b974c817f080 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:13.499971 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 164/335] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:20:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:20:13.505416 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdcff40-a9ba-4662-834e-2d89b5bc0305 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:13.505862 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdcff40-a9ba-4662-834e-2d89b5bc0305 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:13.506032 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdcff40-a9ba-4662-834e-2d89b5bc0305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:13.531650 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-acdcff40-a9ba-4662-834e-2d89b5bc0305 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:20:13.532847 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-acdcff40-a9ba-4662-834e-2d89b5bc0305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:20:13.554349 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:13.569379 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d5caee-50f4-468d-9c1b-cba790aa0c64 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:13.569689 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d5caee-50f4-468d-9c1b-cba790aa0c64 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:13.571668 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d5caee-50f4-468d-9c1b-cba790aa0c64 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:13.571668 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 165/336] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:20:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:20:13.578116 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595bb8e7-acd6-41f2-a389-f226e365e2d6 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:13.578410 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595bb8e7-acd6-41f2-a389-f226e365e2d6 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:13.578747 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595bb8e7-acd6-41f2-a389-f226e365e2d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:13.616644 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-595bb8e7-acd6-41f2-a389-f226e365e2d6 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:20:13.617946 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-595bb8e7-acd6-41f2-a389-f226e365e2d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:20:13.679159 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 172/337] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:20:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 07:20:13.725962 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:13.735100 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 166/338] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:20:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 07:20:13.792928 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92327496-9459-481d-b6af-9f58abc5d763 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:13.793220 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92327496-9459-481d-b6af-9f58abc5d763 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:13.793500 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92327496-9459-481d-b6af-9f58abc5d763 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:13.795075 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 173/339] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:20:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:20:13.800548 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d947079-ef0b-487e-9eb7-86e061509794 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:13.801140 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d947079-ef0b-487e-9eb7-86e061509794 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:13.801418 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d947079-ef0b-487e-9eb7-86e061509794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:13.830588 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-2d947079-ef0b-487e-9eb7-86e061509794 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:20:13.831786 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-2d947079-ef0b-487e-9eb7-86e061509794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:20:13.882097 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2dc902-7126-497d-86cf-cc91685ca49f None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:13.882494 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2dc902-7126-497d-86cf-cc91685ca49f None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:13.882777 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2dc902-7126-497d-86cf-cc91685ca49f None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:13.884252 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 174/340] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:20:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:20:13.911406 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429322d7-917a-43cb-8a50-7ce99cd932c7 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:13.911733 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429322d7-917a-43cb-8a50-7ce99cd932c7 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:13.911966 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429322d7-917a-43cb-8a50-7ce99cd932c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:13.914493 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 167/341] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:20:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 07:20:13.943804 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-429322d7-917a-43cb-8a50-7ce99cd932c7 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:20:13.945062 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-429322d7-917a-43cb-8a50-7ce99cd932c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:20:14.062525 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 175/342] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:20:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 07:20:14.137131 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d947079-ef0b-487e-9eb7-86e061509794 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:14.189872 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429322d7-917a-43cb-8a50-7ce99cd932c7 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:14.296871 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d947079-ef0b-487e-9eb7-86e061509794 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:14.320602 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:14.340919 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:14.341094 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699753ce-0bb5-4368-989f-781dc31747ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:14.341629 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699753ce-0bb5-4368-989f-781dc31747ad None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:14.342513 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699753ce-0bb5-4368-989f-781dc31747ad None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:14.342792 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699753ce-0bb5-4368-989f-781dc31747ad None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:14.354579 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699753ce-0bb5-4368-989f-781dc31747ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:20:14.360058 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-699753ce-0bb5-4368-989f-781dc31747ad None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 07:20:14.360058 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:14.360058 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:14.360058 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:14.360058 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:14.360058 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:14.360058 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:14.360058 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:14.360058 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:14.360058 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:14.360058 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:14.360058 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:14.360058 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:20:14.360058 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:20:14.360058 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:20:14.360058 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:20:14.364876 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:20:14.364876 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:20:14.364876 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:20:14.364876 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:20:14.364876 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:14.364876 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:14.364876 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:14.364876 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:14.364876 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:14.364876 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:14.364876 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:14.364876 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:14.364876 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:14.364876 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:14.364876 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:14.364876 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:14.365706 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:14.365706 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:14.365706 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:14.365706 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:14.365706 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:20:14.365706 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:20:14.365706 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:20:14.365706 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:20:14.365706 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:20:14.365706 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:20:14.365706 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 07:20:14.365706 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:20:14.365706 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 168/343] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Sep 15 07:20:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:14.366856 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429322d7-917a-43cb-8a50-7ce99cd932c7 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:14.375088 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699753ce-0bb5-4368-989f-781dc31747ad None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:14.435166 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc55b4a7-f052-49f3-a0c4-c0b059abe0c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:14.436334 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc55b4a7-f052-49f3-a0c4-c0b059abe0c8 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:14.436612 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc55b4a7-f052-49f3-a0c4-c0b059abe0c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:14.436882 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc55b4a7-f052-49f3-a0c4-c0b059abe0c8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:14.448946 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc55b4a7-f052-49f3-a0c4-c0b059abe0c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:20:14.454374 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-dc55b4a7-f052-49f3-a0c4-c0b059abe0c8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 07:20:14.454374 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:14.454374 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:14.454374 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:14.454374 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:14.454374 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:14.454374 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:14.454374 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:14.454374 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:14.454374 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:14.454374 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:14.454374 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:14.454374 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:20:14.454374 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:20:14.454374 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:20:14.454374 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:20:14.455274 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:20:14.455274 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:20:14.455274 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:20:14.455274 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:20:14.455274 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:14.455274 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:14.455274 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:14.455274 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:14.455274 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:14.455274 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:14.455274 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:14.455274 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:14.455274 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:14.455274 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:14.455274 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:14.455274 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:14.456447 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:14.456447 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:14.456447 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:14.456447 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:14.456447 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:20:14.456447 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:20:14.456447 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:20:14.456447 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:20:14.456447 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:20:14.456447 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:20:14.456447 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 07:20:14.456447 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:20:14.456447 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 176/344] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 15 07:20:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:14.456447 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc55b4a7-f052-49f3-a0c4-c0b059abe0c8 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:14.535219 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699753ce-0bb5-4368-989f-781dc31747ad None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:14.578248 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd52864-613c-4cb6-8aa0-b1d580d711b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:14.579234 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd52864-613c-4cb6-8aa0-b1d580d711b9 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:14.579700 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd52864-613c-4cb6-8aa0-b1d580d711b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:14.580177 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd52864-613c-4cb6-8aa0-b1d580d711b9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:14.595659 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd52864-613c-4cb6-8aa0-b1d580d711b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:20:14.600625 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-abd52864-613c-4cb6-8aa0-b1d580d711b9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 07:20:14.600625 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:14.600625 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:14.600625 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:14.600625 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:14.600625 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:14.600625 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:14.600625 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:14.600625 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:14.600625 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:14.600625 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:14.600625 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:14.600625 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:20:14.600625 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:20:14.600625 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:20:14.600625 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:20:14.601537 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:20:14.601537 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:20:14.601537 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:20:14.601537 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:20:14.601537 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:14.601537 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:14.601537 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:14.601537 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:14.601537 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:14.601537 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:14.601537 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:14.601537 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:14.601537 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:14.601537 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:14.601537 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:14.601537 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:14.602410 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:14.602410 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:14.602410 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:14.602410 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:14.602410 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:20:14.602410 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:20:14.602410 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:20:14.602410 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:20:14.602410 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:20:14.602410 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:20:14.602410 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 07:20:14.602410 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:20:14.602410 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 169/345] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 15 07:20:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:14.603141 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd52864-613c-4cb6-8aa0-b1d580d711b9 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:14.613493 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc55b4a7-f052-49f3-a0c4-c0b059abe0c8 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:14.655138 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:14.667591 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff9ba1b-81fa-46ac-a144-2a11957bd33e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:14.672883 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff9ba1b-81fa-46ac-a144-2a11957bd33e None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:14.672883 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff9ba1b-81fa-46ac-a144-2a11957bd33e None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:14.672883 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff9ba1b-81fa-46ac-a144-2a11957bd33e None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:14.672883 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff9ba1b-81fa-46ac-a144-2a11957bd33e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:14.672883 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff9ba1b-81fa-46ac-a144-2a11957bd33e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:14.672883 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff9ba1b-81fa-46ac-a144-2a11957bd33e None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:14.683297 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 177/346] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 15 07:20:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:14.684439 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff9ba1b-81fa-46ac-a144-2a11957bd33e None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:14.687875 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:14.717661 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd52864-613c-4cb6-8aa0-b1d580d711b9 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:14.759422 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db708ec1-84f1-49d7-93e5-befdd895edbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:14.760035 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db708ec1-84f1-49d7-93e5-befdd895edbb None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:14.760314 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db708ec1-84f1-49d7-93e5-befdd895edbb None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:14.760587 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db708ec1-84f1-49d7-93e5-befdd895edbb None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:14.761254 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db708ec1-84f1-49d7-93e5-befdd895edbb None admin] RBAC: Adding query filter params (name=member) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:14.761556 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db708ec1-84f1-49d7-93e5-befdd895edbb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:14.763058 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db708ec1-84f1-49d7-93e5-befdd895edbb None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:14.773009 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 170/347] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Sep 15 07:20:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:14.773872 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db708ec1-84f1-49d7-93e5-befdd895edbb None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:14.832700 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff9ba1b-81fa-46ac-a144-2a11957bd33e None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:14.880069 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ca7150-8191-4e5d-a1ce-5899236ce46d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:14.881055 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ca7150-8191-4e5d-a1ce-5899236ce46d None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:14.882116 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ca7150-8191-4e5d-a1ce-5899236ce46d None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:14.882624 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ca7150-8191-4e5d-a1ce-5899236ce46d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:14.898273 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db708ec1-84f1-49d7-93e5-befdd895edbb None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:14.899116 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ca7150-8191-4e5d-a1ce-5899236ce46d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:20:14.899976 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:14.906999 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-e2ca7150-8191-4e5d-a1ce-5899236ce46d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 07:20:14.906999 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:14.906999 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:14.906999 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:14.906999 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:14.906999 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:14.906999 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:14.906999 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:14.906999 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:14.906999 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:14.906999 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:14.906999 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:14.906999 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:20:14.906999 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:20:14.906999 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:20:14.906999 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:20:14.908291 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:20:14.908291 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:20:14.908291 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:20:14.908291 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:20:14.908291 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:14.908291 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:14.908291 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:14.908291 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:14.908291 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:14.908291 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:14.908291 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:14.908291 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:14.908291 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:14.908291 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:14.908291 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:14.908291 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:14.909064 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:14.909064 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:14.909064 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:14.909064 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:14.909064 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:20:14.909064 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:20:14.909064 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:20:14.909064 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:20:14.909064 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:20:14.909064 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:20:14.909064 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 07:20:14.909064 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:20:14.913731 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 178/348] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 15 07:20:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:14.915014 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ca7150-8191-4e5d-a1ce-5899236ce46d None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:14.961618 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0da2a95-a7ad-45a8-ac15-d15e1fb48e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:14.962926 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0da2a95-a7ad-45a8-ac15-d15e1fb48e5a None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:14.963549 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0da2a95-a7ad-45a8-ac15-d15e1fb48e5a None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:14.964132 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0da2a95-a7ad-45a8-ac15-d15e1fb48e5a None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:14.964985 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0da2a95-a7ad-45a8-ac15-d15e1fb48e5a None admin] RBAC: Adding query filter params (name=member) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:14.965469 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0da2a95-a7ad-45a8-ac15-d15e1fb48e5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:14.967792 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0da2a95-a7ad-45a8-ac15-d15e1fb48e5a None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:14.977011 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 171/349] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Sep 15 07:20:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:14.978280 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0da2a95-a7ad-45a8-ac15-d15e1fb48e5a None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:14.999977 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:15.051162 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ca7150-8191-4e5d-a1ce-5899236ce46d None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:15.098158 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733043ed-e093-4c58-b322-b27ab4092c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:15.098682 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733043ed-e093-4c58-b322-b27ab4092c31 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:15.098964 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733043ed-e093-4c58-b322-b27ab4092c31 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:15.099815 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733043ed-e093-4c58-b322-b27ab4092c31 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:15.110069 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0da2a95-a7ad-45a8-ac15-d15e1fb48e5a None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:15.158084 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32516f71-4653-47da-89c3-9f828c4a9e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:15.159047 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32516f71-4653-47da-89c3-9f828c4a9e12 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:15.159552 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32516f71-4653-47da-89c3-9f828c4a9e12 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:15.160023 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32516f71-4653-47da-89c3-9f828c4a9e12 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:15.186472 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733043ed-e093-4c58-b322-b27ab4092c31 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:15.188612 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733043ed-e093-4c58-b322-b27ab4092c31 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:15.207760 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:15.258388 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32516f71-4653-47da-89c3-9f828c4a9e12 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:15.259875 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32516f71-4653-47da-89c3-9f828c4a9e12 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:15.266329 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:15.311422 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 179/350] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 15 07:20:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 07:20:15.317692 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733043ed-e093-4c58-b322-b27ab4092c31 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:15.381088 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 172/351] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 15 07:20:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 07:20:15.382697 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32516f71-4653-47da-89c3-9f828c4a9e12 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:15.442919 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733043ed-e093-4c58-b322-b27ab4092c31 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:15.464451 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:15.495270 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe79672d-5337-4b3a-8f76-c91ce3f94c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:15.495877 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe79672d-5337-4b3a-8f76-c91ce3f94c38 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:15.496158 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe79672d-5337-4b3a-8f76-c91ce3f94c38 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:15.496424 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe79672d-5337-4b3a-8f76-c91ce3f94c38 None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:15.497082 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe79672d-5337-4b3a-8f76-c91ce3f94c38 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:15.497388 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe79672d-5337-4b3a-8f76-c91ce3f94c38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:15.499469 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe79672d-5337-4b3a-8f76-c91ce3f94c38 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:15.510135 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 180/352] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Sep 15 07:20:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:15.510989 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe79672d-5337-4b3a-8f76-c91ce3f94c38 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:15.519074 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32516f71-4653-47da-89c3-9f828c4a9e12 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:15.541482 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:15.562213 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ee3991-c38d-4c22-8f3c-95875546d82e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:15.562917 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ee3991-c38d-4c22-8f3c-95875546d82e None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:15.563296 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ee3991-c38d-4c22-8f3c-95875546d82e None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:15.563574 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ee3991-c38d-4c22-8f3c-95875546d82e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:15.651656 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe79672d-5337-4b3a-8f76-c91ce3f94c38 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:15.664110 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ee3991-c38d-4c22-8f3c-95875546d82e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:15.665857 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ee3991-c38d-4c22-8f3c-95875546d82e None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:15.699056 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659d4ac2-c4a8-45b2-9c51-3eb8c437ca37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:15.700095 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659d4ac2-c4a8-45b2-9c51-3eb8c437ca37 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:15.700586 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659d4ac2-c4a8-45b2-9c51-3eb8c437ca37 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:15.701212 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659d4ac2-c4a8-45b2-9c51-3eb8c437ca37 None admin] PATH_INFO: `/v3/projects/659ee9038adf47e585b94881867c18c2` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:15.703329 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659d4ac2-c4a8-45b2-9c51-3eb8c437ca37 None admin] RBAC: Authorizing `identity:get_project(project_id=659ee9038adf47e585b94881867c18c2)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:15.705264 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659d4ac2-c4a8-45b2-9c51-3eb8c437ca37 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:15.711522 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 181/353] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 15 07:20:15 2022] GET /identity/v3/projects/659ee9038adf47e585b94881867c18c2 => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:15.711522 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a1bd42-8206-4f13-898a-d729b05e5a25 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:15.711522 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a1bd42-8206-4f13-898a-d729b05e5a25 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:15.711522 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a1bd42-8206-4f13-898a-d729b05e5a25 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:15.711522 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 182/354] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:20:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:20:15.711522 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a1bd42-8206-4f13-898a-d729b05e5a25 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:15.757538 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:15.807434 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 173/355] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 15 07:20:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 07:20:15.809520 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ee3991-c38d-4c22-8f3c-95875546d82e None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:15.822527 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:15.832489 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a1bd42-8206-4f13-898a-d729b05e5a25 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:15.874776 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ea5bb3-7915-485d-8e0a-a3b41b619e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:15.875415 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ea5bb3-7915-485d-8e0a-a3b41b619e56 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:15.875689 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ea5bb3-7915-485d-8e0a-a3b41b619e56 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:15.875968 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ea5bb3-7915-485d-8e0a-a3b41b619e56 None admin] PATH_INFO: `/v3/projects/659ee9038adf47e585b94881867c18c2` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:15.877203 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ea5bb3-7915-485d-8e0a-a3b41b619e56 None admin] RBAC: Authorizing `identity:get_project(project_id=659ee9038adf47e585b94881867c18c2)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:15.878685 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ea5bb3-7915-485d-8e0a-a3b41b619e56 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:15.880624 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 183/356] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 15 07:20:15 2022] GET /identity/v3/projects/659ee9038adf47e585b94881867c18c2 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:15.881428 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ea5bb3-7915-485d-8e0a-a3b41b619e56 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:15.928988 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ee3991-c38d-4c22-8f3c-95875546d82e None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:15.975898 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06a1380-94a6-475c-9631-3f0c0be4edd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:15.976541 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06a1380-94a6-475c-9631-3f0c0be4edd8 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:15.976822 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06a1380-94a6-475c-9631-3f0c0be4edd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:15.977092 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06a1380-94a6-475c-9631-3f0c0be4edd8 None admin] PATH_INFO: `/v3/users/7e9f8ae5dc9e4704860e97eacca341a7` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:15.979098 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06a1380-94a6-475c-9631-3f0c0be4edd8 None admin] RBAC: Authorizing `identity:get_user(user_id=7e9f8ae5dc9e4704860e97eacca341a7)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:15.980532 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06a1380-94a6-475c-9631-3f0c0be4edd8 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:15.982538 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 174/357] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 15 07:20:15 2022] GET /identity/v3/users/7e9f8ae5dc9e4704860e97eacca341a7 => generated 293 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:15.984391 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee48709-992b-46e8-9145-db21854132fd None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:15.984679 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee48709-992b-46e8-9145-db21854132fd None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:15.984949 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee48709-992b-46e8-9145-db21854132fd None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:15.986247 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 175/358] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:20:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:20:15.987696 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4991e81c-65aa-4d60-a0f9-7ee024fa4289 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:15.987982 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4991e81c-65aa-4d60-a0f9-7ee024fa4289 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:15.988250 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4991e81c-65aa-4d60-a0f9-7ee024fa4289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:16.002235 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ea5bb3-7915-485d-8e0a-a3b41b619e56 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:16.008564 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-4991e81c-65aa-4d60-a0f9-7ee024fa4289 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:20:16.009807 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-4991e81c-65aa-4d60-a0f9-7ee024fa4289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:20:16.039567 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:16.049968 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ae58df-f5f8-485b-8a3a-60616b4e7094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:16.050746 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ae58df-f5f8-485b-8a3a-60616b4e7094 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:16.051029 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ae58df-f5f8-485b-8a3a-60616b4e7094 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:16.051306 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ae58df-f5f8-485b-8a3a-60616b4e7094 None admin] PATH_INFO: `/v3/groups/5c1936c14be64aa491d923cbd86f3988` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:16.059070 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ae58df-f5f8-485b-8a3a-60616b4e7094 None admin] RBAC: Authorizing `identity:get_group(group_id=5c1936c14be64aa491d923cbd86f3988)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:16.060113 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ae58df-f5f8-485b-8a3a-60616b4e7094 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:16.065664 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 184/359] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Sep 15 07:20:15 2022] GET /identity/v3/groups/5c1936c14be64aa491d923cbd86f3988 => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:16.065664 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ae58df-f5f8-485b-8a3a-60616b4e7094 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:16.092780 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:16.094770 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 176/360] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:20:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 07:20:16.096060 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4991e81c-65aa-4d60-a0f9-7ee024fa4289 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:16.202898 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ae58df-f5f8-485b-8a3a-60616b4e7094 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:16.230194 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4991e81c-65aa-4d60-a0f9-7ee024fa4289 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:16.250065 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a997ee-6224-45ec-964d-9a51feecb347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:16.250708 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a997ee-6224-45ec-964d-9a51feecb347 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:16.250983 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a997ee-6224-45ec-964d-9a51feecb347 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:16.251250 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a997ee-6224-45ec-964d-9a51feecb347 None admin] PATH_INFO: `/v3/projects/91757d436677474ab5f1027263238244` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:16.252462 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a997ee-6224-45ec-964d-9a51feecb347 None admin] RBAC: Authorizing `identity:get_project(project_id=91757d436677474ab5f1027263238244)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:16.254112 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a997ee-6224-45ec-964d-9a51feecb347 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:16.255978 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 185/361] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 15 07:20:16 2022] GET /identity/v3/projects/91757d436677474ab5f1027263238244 => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:16.256939 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a997ee-6224-45ec-964d-9a51feecb347 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:16.281966 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4d179f-b339-452d-86cf-65ce43538133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:16.281966 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4d179f-b339-452d-86cf-65ce43538133 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:16.281966 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4d179f-b339-452d-86cf-65ce43538133 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:16.281966 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4d179f-b339-452d-86cf-65ce43538133 None admin] PATH_INFO: `/v3/groups/5c1936c14be64aa491d923cbd86f3988` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:16.283113 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4d179f-b339-452d-86cf-65ce43538133 None admin] RBAC: Authorizing `identity:get_group(group_id=5c1936c14be64aa491d923cbd86f3988)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:16.284533 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4d179f-b339-452d-86cf-65ce43538133 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:16.287019 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 177/362] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Sep 15 07:20:16 2022] GET /identity/v3/groups/5c1936c14be64aa491d923cbd86f3988 => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:16.287647 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03e881a-f306-4809-9d08-a4ca1ec32fcf None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:16.287961 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03e881a-f306-4809-9d08-a4ca1ec32fcf None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:16.288639 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03e881a-f306-4809-9d08-a4ca1ec32fcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:16.309535 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-d03e881a-f306-4809-9d08-a4ca1ec32fcf None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:20:16.311538 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-d03e881a-f306-4809-9d08-a4ca1ec32fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:20:16.345260 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:16.381871 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:16.392465 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a997ee-6224-45ec-964d-9a51feecb347 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:16.401775 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 178/363] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:20:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 07:20:16.402639 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03e881a-f306-4809-9d08-a4ca1ec32fcf None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:16.432929 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b12eb7-ee16-479d-af14-f8dce07b38e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:16.433536 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b12eb7-ee16-479d-af14-f8dce07b38e7 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:16.433862 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b12eb7-ee16-479d-af14-f8dce07b38e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:16.434129 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b12eb7-ee16-479d-af14-f8dce07b38e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:16.434843 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b12eb7-ee16-479d-af14-f8dce07b38e7 None admin] RBAC: Adding query filter params (role.id=81b8a5256ba14184beb300947fbfe335, user.id=7e9f8ae5dc9e4704860e97eacca341a7, scope.system=all) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:16.435150 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b12eb7-ee16-479d-af14-f8dce07b38e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:16.437405 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b12eb7-ee16-479d-af14-f8dce07b38e7 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:16.449331 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 186/364] 10.222.0.42 () {56 vars in 1308 bytes} [Thu Sep 15 07:20:16 2022] GET /identity/v3/role_assignments?user.id=7e9f8ae5dc9e4704860e97eacca341a7&scope.system=all&role.id=81b8a5256ba14184beb300947fbfe335 => generated 510 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:16.453709 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b12eb7-ee16-479d-af14-f8dce07b38e7 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:16.535752 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03e881a-f306-4809-9d08-a4ca1ec32fcf None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:16.585907 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366d5d29-29c6-4b86-8f78-954022bb084a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:16.587339 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366d5d29-29c6-4b86-8f78-954022bb084a None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:16.587790 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366d5d29-29c6-4b86-8f78-954022bb084a None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:16.588204 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366d5d29-29c6-4b86-8f78-954022bb084a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:16.589085 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366d5d29-29c6-4b86-8f78-954022bb084a None admin] RBAC: Adding query filter params (group.id=5c1936c14be64aa491d923cbd86f3988, role.id=a0fae926490542b4977d1ecf27fcece4, scope.project.id=659ee9038adf47e585b94881867c18c2) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:16.589558 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366d5d29-29c6-4b86-8f78-954022bb084a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:16.592506 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366d5d29-29c6-4b86-8f78-954022bb084a None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:16.601027 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 179/365] 10.222.0.42 () {56 vars in 1376 bytes} [Thu Sep 15 07:20:16 2022] GET /identity/v3/role_assignments?group.id=5c1936c14be64aa491d923cbd86f3988&scope.project.id=659ee9038adf47e585b94881867c18c2&role.id=a0fae926490542b4977d1ecf27fcece4 => generated 260 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:16.602017 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b12eb7-ee16-479d-af14-f8dce07b38e7 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:16.602421 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366d5d29-29c6-4b86-8f78-954022bb084a None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:16.660517 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:16.673328 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a2974d-7ca3-44bb-bb1a-03b0fd60c891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:16.674025 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a2974d-7ca3-44bb-bb1a-03b0fd60c891 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:16.674306 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a2974d-7ca3-44bb-bb1a-03b0fd60c891 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:16.674576 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a2974d-7ca3-44bb-bb1a-03b0fd60c891 None admin] PATH_INFO: `/v3/groups/5c1936c14be64aa491d923cbd86f3988` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:16.676422 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a2974d-7ca3-44bb-bb1a-03b0fd60c891 None admin] RBAC: Authorizing `identity:get_group(group_id=5c1936c14be64aa491d923cbd86f3988)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:16.677922 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a2974d-7ca3-44bb-bb1a-03b0fd60c891 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:16.680564 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 187/366] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Sep 15 07:20:16 2022] GET /identity/v3/groups/5c1936c14be64aa491d923cbd86f3988 => generated 226 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:16.680692 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a2974d-7ca3-44bb-bb1a-03b0fd60c891 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:16.714136 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:16.781981 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366d5d29-29c6-4b86-8f78-954022bb084a None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:16.824846 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fa0cd9-a7e1-4d46-9c13-09502350c94f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:16.825593 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fa0cd9-a7e1-4d46-9c13-09502350c94f None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:16.825874 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fa0cd9-a7e1-4d46-9c13-09502350c94f None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:16.826144 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fa0cd9-a7e1-4d46-9c13-09502350c94f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:16.827477 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fa0cd9-a7e1-4d46-9c13-09502350c94f None admin] RBAC: Adding query filter params (group.id=5c1936c14be64aa491d923cbd86f3988, role.id=053e12af84e943e699460a9d76c2072b, scope.project.id=659ee9038adf47e585b94881867c18c2) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:16.827783 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fa0cd9-a7e1-4d46-9c13-09502350c94f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:16.829357 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fa0cd9-a7e1-4d46-9c13-09502350c94f None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:16.832104 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a2974d-7ca3-44bb-bb1a-03b0fd60c891 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:16.836498 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 180/367] 10.222.0.42 () {56 vars in 1376 bytes} [Thu Sep 15 07:20:16 2022] GET /identity/v3/role_assignments?group.id=5c1936c14be64aa491d923cbd86f3988&scope.project.id=659ee9038adf47e585b94881867c18c2&role.id=053e12af84e943e699460a9d76c2072b => generated 260 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:16.837432 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fa0cd9-a7e1-4d46-9c13-09502350c94f None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:16.874454 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f3b6ff-18ea-4f61-acf1-99a597cb688d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:16.875427 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f3b6ff-18ea-4f61-acf1-99a597cb688d None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:16.875880 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f3b6ff-18ea-4f61-acf1-99a597cb688d None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:16.876326 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f3b6ff-18ea-4f61-acf1-99a597cb688d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:16.889266 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f3b6ff-18ea-4f61-acf1-99a597cb688d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:20:16.894786 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-f3f3b6ff-18ea-4f61-acf1-99a597cb688d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 07:20:16.894786 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:16.894786 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:16.894786 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:16.894786 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:16.894786 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:16.894786 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:16.894786 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:16.894786 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:16.894786 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:16.894786 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:16.894786 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:16.894786 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:20:16.894786 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:20:16.894786 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:20:16.894786 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:20:16.895733 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:20:16.895733 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:20:16.895733 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:20:16.895733 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:20:16.895733 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:16.895733 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:16.895733 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:16.895733 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:16.895733 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:16.895733 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:16.895733 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:16.895733 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:16.895733 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:16.895733 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:16.895733 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:16.895733 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:16.896524 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:16.896524 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:16.896524 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:16.896524 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:16.896524 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:20:16.896524 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:20:16.896524 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:20:16.896524 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:20:16.896524 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:20:16.896524 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:20:16.896524 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 07:20:16.896524 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:20:16.896524 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 188/368] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Sep 15 07:20:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:16.897194 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f3b6ff-18ea-4f61-acf1-99a597cb688d None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:16.973009 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:16.986596 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fa0cd9-a7e1-4d46-9c13-09502350c94f None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:17.016933 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:17.033613 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50664124-4621-4701-9120-9dcb64d117c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:17.034319 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50664124-4621-4701-9120-9dcb64d117c4 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:17.034688 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50664124-4621-4701-9120-9dcb64d117c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:17.035030 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50664124-4621-4701-9120-9dcb64d117c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:17.035773 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50664124-4621-4701-9120-9dcb64d117c4 None admin] RBAC: Adding query filter params (group.id=5c1936c14be64aa491d923cbd86f3988, role.id=053e12af84e943e699460a9d76c2072b, scope.project.id=91757d436677474ab5f1027263238244) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:17.036079 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50664124-4621-4701-9120-9dcb64d117c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:17.037509 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50664124-4621-4701-9120-9dcb64d117c4 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:17.044469 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 181/369] 10.222.0.42 () {56 vars in 1376 bytes} [Thu Sep 15 07:20:16 2022] GET /identity/v3/role_assignments?group.id=5c1936c14be64aa491d923cbd86f3988&scope.project.id=91757d436677474ab5f1027263238244&role.id=053e12af84e943e699460a9d76c2072b => generated 260 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:17.046033 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50664124-4621-4701-9120-9dcb64d117c4 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:17.047544 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f3b6ff-18ea-4f61-acf1-99a597cb688d None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:17.096628 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af58cee-de2e-4950-9793-e7422caa0881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:17.097386 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af58cee-de2e-4950-9793-e7422caa0881 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:17.097697 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af58cee-de2e-4950-9793-e7422caa0881 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:17.097970 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af58cee-de2e-4950-9793-e7422caa0881 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:17.112390 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af58cee-de2e-4950-9793-e7422caa0881 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:20:17.118647 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-5af58cee-de2e-4950-9793-e7422caa0881 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 07:20:17.118647 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:17.118647 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:17.118647 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:17.118647 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:17.118647 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:17.118647 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:17.118647 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:17.118647 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:17.118647 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:17.118647 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:17.118647 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:17.118647 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:20:17.118647 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:20:17.118647 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:20:17.118647 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:20:17.119594 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:20:17.119594 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:20:17.119594 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:20:17.119594 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:20:17.119594 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:17.119594 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:17.119594 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:17.119594 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:17.119594 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:17.119594 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:17.119594 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:17.119594 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:17.119594 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:17.119594 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:17.119594 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:17.119594 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:17.120401 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:17.120401 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:17.120401 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:17.120401 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:17.120401 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:20:17.120401 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:20:17.120401 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:20:17.120401 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:20:17.120401 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:20:17.120401 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:20:17.120401 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 07:20:17.120401 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:20:17.120401 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 189/370] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 15 07:20:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:17.127764 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af58cee-de2e-4950-9793-e7422caa0881 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:17.228336 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50664124-4621-4701-9120-9dcb64d117c4 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:17.291040 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2429ac93-5a60-453d-8dee-808b1b7abb3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:17.291819 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2429ac93-5a60-453d-8dee-808b1b7abb3a None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:17.292104 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2429ac93-5a60-453d-8dee-808b1b7abb3a None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:17.292371 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2429ac93-5a60-453d-8dee-808b1b7abb3a None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:17.293039 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2429ac93-5a60-453d-8dee-808b1b7abb3a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:17.293341 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2429ac93-5a60-453d-8dee-808b1b7abb3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:17.294739 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2429ac93-5a60-453d-8dee-808b1b7abb3a None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:17.307380 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 182/371] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 15 07:20:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:17.312576 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af58cee-de2e-4950-9793-e7422caa0881 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:17.314053 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2429ac93-5a60-453d-8dee-808b1b7abb3a None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:17.335137 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:17.355479 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4e4093-c501-45b6-95d8-1930cdd07f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:17.356203 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4e4093-c501-45b6-95d8-1930cdd07f58 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:17.356483 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4e4093-c501-45b6-95d8-1930cdd07f58 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:17.356856 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4e4093-c501-45b6-95d8-1930cdd07f58 None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:17.358990 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4e4093-c501-45b6-95d8-1930cdd07f58 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:17.359312 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4e4093-c501-45b6-95d8-1930cdd07f58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:17.360715 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4e4093-c501-45b6-95d8-1930cdd07f58 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:17.367783 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:17.372688 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 190/372] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 15 07:20:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:17.379225 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4e4093-c501-45b6-95d8-1930cdd07f58 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:17.436220 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2429ac93-5a60-453d-8dee-808b1b7abb3a None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:17.474059 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26d296f-e27a-4373-90a2-e37cde6258f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:17.474668 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26d296f-e27a-4373-90a2-e37cde6258f5 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:17.474948 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26d296f-e27a-4373-90a2-e37cde6258f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:17.475216 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26d296f-e27a-4373-90a2-e37cde6258f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:17.556320 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26d296f-e27a-4373-90a2-e37cde6258f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:17.558211 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26d296f-e27a-4373-90a2-e37cde6258f5 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:17.566204 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4e4093-c501-45b6-95d8-1930cdd07f58 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:17.576883 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:17.649372 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9f3d9b-6892-4d6f-b35c-8bbea46b289c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:17.650048 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9f3d9b-6892-4d6f-b35c-8bbea46b289c None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:17.650332 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9f3d9b-6892-4d6f-b35c-8bbea46b289c None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:17.650623 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9f3d9b-6892-4d6f-b35c-8bbea46b289c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:17.677361 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 183/373] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 15 07:20:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 15 07:20:17.683539 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26d296f-e27a-4373-90a2-e37cde6258f5 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:17.798076 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26d296f-e27a-4373-90a2-e37cde6258f5 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:17.815624 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:17.817123 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:17.821910 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9f3d9b-6892-4d6f-b35c-8bbea46b289c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:17.823430 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9f3d9b-6892-4d6f-b35c-8bbea46b289c None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:17.838976 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104b06ae-3752-4caa-a90d-56c5b66dc178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:17.839619 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b06ae-3752-4caa-a90d-56c5b66dc178 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:17.839992 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b06ae-3752-4caa-a90d-56c5b66dc178 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:17.840270 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b06ae-3752-4caa-a90d-56c5b66dc178 None admin] PATH_INFO: `/v3/projects/91757d436677474ab5f1027263238244` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:17.841414 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104b06ae-3752-4caa-a90d-56c5b66dc178 None admin] RBAC: Authorizing `identity:get_project(project_id=91757d436677474ab5f1027263238244)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:17.843080 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104b06ae-3752-4caa-a90d-56c5b66dc178 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:17.845070 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 184/374] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 15 07:20:17 2022] GET /identity/v3/projects/91757d436677474ab5f1027263238244 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:20:17.851062 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104b06ae-3752-4caa-a90d-56c5b66dc178 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:17.978694 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104b06ae-3752-4caa-a90d-56c5b66dc178 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:18.019203 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7be941-4a15-46ad-a6f0-f6c2b64b1eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:18.020110 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7be941-4a15-46ad-a6f0-f6c2b64b1eaa None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:18.020395 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7be941-4a15-46ad-a6f0-f6c2b64b1eaa None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:18.020766 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7be941-4a15-46ad-a6f0-f6c2b64b1eaa None admin] PATH_INFO: `/v3/groups/5c1936c14be64aa491d923cbd86f3988` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:18.021923 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7be941-4a15-46ad-a6f0-f6c2b64b1eaa None admin] RBAC: Authorizing `identity:get_group(group_id=5c1936c14be64aa491d923cbd86f3988)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:18.023482 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7be941-4a15-46ad-a6f0-f6c2b64b1eaa None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:18.025366 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 185/375] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Sep 15 07:20:17 2022] GET /identity/v3/groups/5c1936c14be64aa491d923cbd86f3988 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:18.031767 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7be941-4a15-46ad-a6f0-f6c2b64b1eaa None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:18.032568 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 191/376] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 15 07:20:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 654 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 15 07:20:18.037795 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9f3d9b-6892-4d6f-b35c-8bbea46b289c None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:18.099393 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:18.159935 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7be941-4a15-46ad-a6f0-f6c2b64b1eaa None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:18.200367 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb2c3ce-b775-4def-b2ee-c346daacd249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:18.200987 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb2c3ce-b775-4def-b2ee-c346daacd249 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:18.201304 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb2c3ce-b775-4def-b2ee-c346daacd249 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:18.202288 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb2c3ce-b775-4def-b2ee-c346daacd249 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:18.203218 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb2c3ce-b775-4def-b2ee-c346daacd249 None admin] RBAC: Adding query filter params (group.id=5c1936c14be64aa491d923cbd86f3988, role.id=a0fae926490542b4977d1ecf27fcece4, scope.project.id=91757d436677474ab5f1027263238244) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:18.203535 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb2c3ce-b775-4def-b2ee-c346daacd249 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:18.205024 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb2c3ce-b775-4def-b2ee-c346daacd249 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:18.211445 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 186/377] 10.222.0.42 () {56 vars in 1376 bytes} [Thu Sep 15 07:20:18 2022] GET /identity/v3/role_assignments?group.id=5c1936c14be64aa491d923cbd86f3988&scope.project.id=91757d436677474ab5f1027263238244&role.id=a0fae926490542b4977d1ecf27fcece4 => generated 260 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:20:18.240996 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:18.244111 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9f3d9b-6892-4d6f-b35c-8bbea46b289c None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:18.294363 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f933dd-6216-4df0-8984-684f70a767a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:18.295054 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f933dd-6216-4df0-8984-684f70a767a3 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:18.295540 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f933dd-6216-4df0-8984-684f70a767a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:18.295866 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f933dd-6216-4df0-8984-684f70a767a3 None admin] PATH_INFO: `/v3/projects/918580af0a81452f8da95d60256aaea0` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:18.302568 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f933dd-6216-4df0-8984-684f70a767a3 None admin] RBAC: Authorizing `identity:get_project(project_id=918580af0a81452f8da95d60256aaea0)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:18.304191 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f933dd-6216-4df0-8984-684f70a767a3 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:18.306232 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 192/378] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 15 07:20:18 2022] GET /identity/v3/projects/918580af0a81452f8da95d60256aaea0 => generated 344 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:20:18.312881 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb2c3ce-b775-4def-b2ee-c346daacd249 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:18.455549 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb2c3ce-b775-4def-b2ee-c346daacd249 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:18.471455 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:18.494374 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d723a9ec-bc15-4edb-b324-6e1515a30321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:18.495036 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d723a9ec-bc15-4edb-b324-6e1515a30321 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:18.495327 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d723a9ec-bc15-4edb-b324-6e1515a30321 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:18.495601 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d723a9ec-bc15-4edb-b324-6e1515a30321 None admin] PATH_INFO: `/v3/groups/d47a44a719af4dcd80064b386ac52a77` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:18.503893 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d723a9ec-bc15-4edb-b324-6e1515a30321 None admin] RBAC: Authorizing `identity:get_group(group_id=d47a44a719af4dcd80064b386ac52a77)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:18.505384 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d723a9ec-bc15-4edb-b324-6e1515a30321 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:18.507536 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 187/379] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Sep 15 07:20:18 2022] GET /identity/v3/groups/d47a44a719af4dcd80064b386ac52a77 => generated 229 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:18.513169 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f933dd-6216-4df0-8984-684f70a767a3 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:18.637327 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f933dd-6216-4df0-8984-684f70a767a3 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:18.684216 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8e04e7-718c-4edb-91fc-c6a110725b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:18.684829 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8e04e7-718c-4edb-91fc-c6a110725b34 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:18.685107 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8e04e7-718c-4edb-91fc-c6a110725b34 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:18.685377 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8e04e7-718c-4edb-91fc-c6a110725b34 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:18.686701 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8e04e7-718c-4edb-91fc-c6a110725b34 None admin] RBAC: Adding query filter params (group.id=d47a44a719af4dcd80064b386ac52a77, role.id=6bbb643e0d8d4bc0a9b27c26edf59216, scope.project.id=918580af0a81452f8da95d60256aaea0) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:18.687014 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8e04e7-718c-4edb-91fc-c6a110725b34 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:18.688522 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8e04e7-718c-4edb-91fc-c6a110725b34 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:18.694770 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 193/380] 10.222.0.42 () {56 vars in 1376 bytes} [Thu Sep 15 07:20:18 2022] GET /identity/v3/role_assignments?group.id=d47a44a719af4dcd80064b386ac52a77&scope.project.id=918580af0a81452f8da95d60256aaea0&role.id=6bbb643e0d8d4bc0a9b27c26edf59216 => generated 260 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:20:19.533718 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed00e662-d82d-4721-b16e-082c8e188aa5 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:19.534740 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed00e662-d82d-4721-b16e-082c8e188aa5 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:19.535248 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed00e662-d82d-4721-b16e-082c8e188aa5 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:19.537084 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 188/381] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:20:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:20:19.542671 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6092e356-9755-49ae-9786-cbc1ce662985 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:19.551537 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6092e356-9755-49ae-9786-cbc1ce662985 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:19.551537 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6092e356-9755-49ae-9786-cbc1ce662985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:19.567065 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-6092e356-9755-49ae-9786-cbc1ce662985 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:20:19.568575 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-6092e356-9755-49ae-9786-cbc1ce662985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:20:19.595923 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de81aac7-d431-4c7c-88c6-36099d0e69d8 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:19.596565 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de81aac7-d431-4c7c-88c6-36099d0e69d8 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:19.597047 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de81aac7-d431-4c7c-88c6-36099d0e69d8 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:19.598942 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 189/382] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:20:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:20:19.604025 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:19.605788 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ba34c4-b320-47a1-8996-74cf39413e1b None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:19.606366 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ba34c4-b320-47a1-8996-74cf39413e1b None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:19.606851 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ba34c4-b320-47a1-8996-74cf39413e1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:19.638199 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-46ba34c4-b320-47a1-8996-74cf39413e1b None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:20:19.641774 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-46ba34c4-b320-47a1-8996-74cf39413e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:20:19.682779 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 194/383] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:20:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 07:20:19.742554 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 190/384] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:20:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 07:20:20.007243 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6092e356-9755-49ae-9786-cbc1ce662985 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:20.108570 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ba34c4-b320-47a1-8996-74cf39413e1b None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:20.173870 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6092e356-9755-49ae-9786-cbc1ce662985 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:20.221325 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bfeb6f-aefe-4a3c-bfa7-bb69b4dc7017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:20.222478 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bfeb6f-aefe-4a3c-bfa7-bb69b4dc7017 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:20.222917 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bfeb6f-aefe-4a3c-bfa7-bb69b4dc7017 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:20.223458 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bfeb6f-aefe-4a3c-bfa7-bb69b4dc7017 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:20.236578 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bfeb6f-aefe-4a3c-bfa7-bb69b4dc7017 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:20:20.240886 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ba34c4-b320-47a1-8996-74cf39413e1b None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:20.242608 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-c4bfeb6f-aefe-4a3c-bfa7-bb69b4dc7017 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 07:20:20.242608 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:20.242608 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:20.242608 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:20.242608 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:20.242608 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:20.242608 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:20.242608 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:20.242608 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:20.242608 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:20.242608 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:20.242608 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:20.242608 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:20:20.242608 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:20:20.242608 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:20:20.242608 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:20:20.243718 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:20:20.243718 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:20:20.243718 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:20:20.243718 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:20:20.243718 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:20.243718 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:20.243718 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:20.243718 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:20.243718 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:20.243718 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:20.243718 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:20.243718 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:20.243718 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:20.243718 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:20.243718 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:20.243718 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:20.244587 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:20.244587 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:20.244587 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:20.244587 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:20.244587 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:20:20.244587 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:20:20.244587 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:20:20.244587 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:20:20.244587 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:20:20.244587 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:20:20.244587 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 07:20:20.244587 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:20:20.244587 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 195/385] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Sep 15 07:20:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:20.245506 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66ac962-5e55-42fa-9f04-5e95c1fc2f3c None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:20.246010 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66ac962-5e55-42fa-9f04-5e95c1fc2f3c None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:20.246465 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66ac962-5e55-42fa-9f04-5e95c1fc2f3c None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:20.254034 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 196/386] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:20:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:20:20.255127 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66ac962-5e55-42fa-9f04-5e95c1fc2f3c None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:20.256005 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:20.285172 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc36a00e-0013-4f47-a6b0-fbe408d90a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:20.286200 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:20.287133 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc36a00e-0013-4f47-a6b0-fbe408d90a91 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:20.287888 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc36a00e-0013-4f47-a6b0-fbe408d90a91 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:20.288253 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc36a00e-0013-4f47-a6b0-fbe408d90a91 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:20.300818 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc36a00e-0013-4f47-a6b0-fbe408d90a91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:20:20.307403 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-bc36a00e-0013-4f47-a6b0-fbe408d90a91 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 07:20:20.307403 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:20.307403 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:20.307403 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:20.307403 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:20.307403 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:20.307403 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:20.307403 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:20.307403 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:20.307403 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:20.307403 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:20.307403 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:20.307403 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:20:20.307403 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:20:20.307403 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:20:20.307403 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:20:20.308270 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:20:20.308270 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:20:20.308270 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:20:20.308270 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:20:20.308270 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:20.308270 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:20.308270 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:20.308270 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:20.308270 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:20.308270 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:20.308270 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:20.308270 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:20.308270 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:20.308270 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:20.308270 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:20.308270 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:20.309052 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:20.309052 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:20.309052 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:20.309052 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:20.309052 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:20:20.309052 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:20:20.309052 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:20:20.309052 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:20:20.309052 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:20:20.309052 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:20:20.309052 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 07:20:20.309052 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:20:20.311809 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 191/387] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 15 07:20:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:20.313183 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12bbc43-a5ff-4027-923c-e81bdefe0ebc None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:20.314048 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12bbc43-a5ff-4027-923c-e81bdefe0ebc None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:20.314489 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12bbc43-a5ff-4027-923c-e81bdefe0ebc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:20.335015 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-e12bbc43-a5ff-4027-923c-e81bdefe0ebc None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:20:20.336424 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-e12bbc43-a5ff-4027-923c-e81bdefe0ebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:20:20.382937 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66ac962-5e55-42fa-9f04-5e95c1fc2f3c None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:20.417726 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 192/388] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:20:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 07:20:20.418621 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12bbc43-a5ff-4027-923c-e81bdefe0ebc None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:20.431858 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2bf264-e150-458d-bed5-c78de96ef5dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:20.432748 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2bf264-e150-458d-bed5-c78de96ef5dd None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:20.433081 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2bf264-e150-458d-bed5-c78de96ef5dd None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:20.433360 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2bf264-e150-458d-bed5-c78de96ef5dd None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:20.434067 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2bf264-e150-458d-bed5-c78de96ef5dd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:20.434372 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2bf264-e150-458d-bed5-c78de96ef5dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:20.438121 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2bf264-e150-458d-bed5-c78de96ef5dd None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:20.452826 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 197/389] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 15 07:20:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:20.459221 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2bf264-e150-458d-bed5-c78de96ef5dd None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:20.558256 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:20.566529 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12bbc43-a5ff-4027-923c-e81bdefe0ebc None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:20.636997 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:20.640249 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98c4db1-6fd8-476c-8682-123eedaa6b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:20.640876 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98c4db1-6fd8-476c-8682-123eedaa6b72 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:20.641160 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98c4db1-6fd8-476c-8682-123eedaa6b72 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:20.641489 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98c4db1-6fd8-476c-8682-123eedaa6b72 None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:20.642194 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98c4db1-6fd8-476c-8682-123eedaa6b72 None admin] RBAC: Adding query filter params (name=member) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:20.642499 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98c4db1-6fd8-476c-8682-123eedaa6b72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:20.643873 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98c4db1-6fd8-476c-8682-123eedaa6b72 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:20.663738 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 193/390] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Sep 15 07:20:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:20.664477 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2bf264-e150-458d-bed5-c78de96ef5dd None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:20.668549 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98c4db1-6fd8-476c-8682-123eedaa6b72 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:20.729764 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e90ca43-33d9-4bf4-8a32-cf9f0c675e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:20.730460 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e90ca43-33d9-4bf4-8a32-cf9f0c675e5e None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:20.730747 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e90ca43-33d9-4bf4-8a32-cf9f0c675e5e None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:20.731025 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e90ca43-33d9-4bf4-8a32-cf9f0c675e5e None admin] PATH_INFO: `/v3/groups/5c1936c14be64aa491d923cbd86f3988` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:20.732154 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e90ca43-33d9-4bf4-8a32-cf9f0c675e5e None admin] RBAC: Authorizing `identity:get_group(group_id=5c1936c14be64aa491d923cbd86f3988)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:20.734224 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e90ca43-33d9-4bf4-8a32-cf9f0c675e5e None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:20.740094 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 198/391] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Sep 15 07:20:20 2022] GET /identity/v3/groups/5c1936c14be64aa491d923cbd86f3988 => generated 226 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:20.740094 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e90ca43-33d9-4bf4-8a32-cf9f0c675e5e None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:20.814366 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98c4db1-6fd8-476c-8682-123eedaa6b72 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:20.843661 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e90ca43-33d9-4bf4-8a32-cf9f0c675e5e None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:20.852269 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d7f6fb-cb44-4ec2-bee7-e0c54b3032f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:20.853200 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d7f6fb-cb44-4ec2-bee7-e0c54b3032f2 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:20.853749 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d7f6fb-cb44-4ec2-bee7-e0c54b3032f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:20.854239 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d7f6fb-cb44-4ec2-bee7-e0c54b3032f2 None admin] PATH_INFO: `/v3/groups/5c1936c14be64aa491d923cbd86f3988` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:20.855686 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d7f6fb-cb44-4ec2-bee7-e0c54b3032f2 None admin] RBAC: Authorizing `identity:get_group(group_id=5c1936c14be64aa491d923cbd86f3988)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:20.857506 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d7f6fb-cb44-4ec2-bee7-e0c54b3032f2 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:20.860534 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 194/392] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Sep 15 07:20:20 2022] GET /identity/v3/groups/5c1936c14be64aa491d923cbd86f3988 => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:20.861761 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d7f6fb-cb44-4ec2-bee7-e0c54b3032f2 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:20.880478 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9f3e0d-6b13-4317-8ead-30e5d9b25726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:20.881669 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9f3e0d-6b13-4317-8ead-30e5d9b25726 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:20.882344 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9f3e0d-6b13-4317-8ead-30e5d9b25726 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:20.882996 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9f3e0d-6b13-4317-8ead-30e5d9b25726 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:20.897197 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9f3e0d-6b13-4317-8ead-30e5d9b25726 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:20:20.898879 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:20.903297 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-1b9f3e0d-6b13-4317-8ead-30e5d9b25726 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 07:20:20.903297 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:20.903297 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:20.903297 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:20.903297 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:20.903297 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:20.903297 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:20.903297 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:20.903297 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:20.903297 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:20.903297 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:20.903297 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:20.903297 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:20:20.903297 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:20:20.903297 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:20:20.903297 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:20:20.904273 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:20:20.904273 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:20:20.904273 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:20:20.904273 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:20:20.904273 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:20.904273 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:20.904273 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:20.904273 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:20.904273 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:20.904273 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:20.904273 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:20.904273 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:20.904273 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:20.904273 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:20.904273 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:20.904273 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:20.905596 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:20.905596 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:20.905596 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:20.905596 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:20.905596 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:20:20.905596 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:20:20.905596 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:20:20.905596 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:20:20.905596 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:20:20.905596 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:20:20.905596 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 07:20:20.905596 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:20:20.908019 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 199/393] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 15 07:20:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:20.909143 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9f3e0d-6b13-4317-8ead-30e5d9b25726 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:20.925707 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:20.987800 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d7f6fb-cb44-4ec2-bee7-e0c54b3032f2 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:21.026663 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089c0657-6f94-4d0a-bf74-57c4e8eeb285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:21.027345 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089c0657-6f94-4d0a-bf74-57c4e8eeb285 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:21.027623 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089c0657-6f94-4d0a-bf74-57c4e8eeb285 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:21.028005 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089c0657-6f94-4d0a-bf74-57c4e8eeb285 None admin] PATH_INFO: `/v3/projects/659ee9038adf47e585b94881867c18c2` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:21.029250 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089c0657-6f94-4d0a-bf74-57c4e8eeb285 None admin] RBAC: Authorizing `identity:get_project(project_id=659ee9038adf47e585b94881867c18c2)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:21.031439 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089c0657-6f94-4d0a-bf74-57c4e8eeb285 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:21.032094 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9f3e0d-6b13-4317-8ead-30e5d9b25726 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:21.033745 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 195/394] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 15 07:20:20 2022] GET /identity/v3/projects/659ee9038adf47e585b94881867c18c2 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:21.034064 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089c0657-6f94-4d0a-bf74-57c4e8eeb285 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:21.069950 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aa1975-4188-4464-a3de-96e060459bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:21.070511 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aa1975-4188-4464-a3de-96e060459bd7 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:21.070794 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aa1975-4188-4464-a3de-96e060459bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:21.071070 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aa1975-4188-4464-a3de-96e060459bd7 None admin] PATH_INFO: `/v3/projects/659ee9038adf47e585b94881867c18c2` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:21.072216 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48aa1975-4188-4464-a3de-96e060459bd7 None admin] RBAC: Authorizing `identity:get_project(project_id=659ee9038adf47e585b94881867c18c2)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:21.073717 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48aa1975-4188-4464-a3de-96e060459bd7 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:21.080448 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 200/395] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 15 07:20:20 2022] GET /identity/v3/projects/659ee9038adf47e585b94881867c18c2 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:21.080448 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f18b6c-134e-4e82-a1aa-69984c80a677 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:21.080448 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f18b6c-134e-4e82-a1aa-69984c80a677 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:21.080448 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f18b6c-134e-4e82-a1aa-69984c80a677 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:21.080448 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 201/396] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:20:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:20:21.080448 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f18b6c-134e-4e82-a1aa-69984c80a677 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:21.132392 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:21.138471 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089c0657-6f94-4d0a-bf74-57c4e8eeb285 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:21.171456 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:21.176407 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3cc430-965c-4de1-8444-e851c5b5ed86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:21.184772 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3cc430-965c-4de1-8444-e851c5b5ed86 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:21.184772 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3cc430-965c-4de1-8444-e851c5b5ed86 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:21.184772 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3cc430-965c-4de1-8444-e851c5b5ed86 None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:21.184772 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3cc430-965c-4de1-8444-e851c5b5ed86 None admin] RBAC: Adding query filter params (name=member) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:21.184772 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3cc430-965c-4de1-8444-e851c5b5ed86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:21.184772 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3cc430-965c-4de1-8444-e851c5b5ed86 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:21.189780 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 196/397] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Sep 15 07:20:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:21.192636 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f18b6c-134e-4e82-a1aa-69984c80a677 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:21.194860 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3cc430-965c-4de1-8444-e851c5b5ed86 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:21.227655 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4bfae1-2652-4f66-9fc4-078cb4b2facc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:21.228335 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4bfae1-2652-4f66-9fc4-078cb4b2facc None admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:21.228627 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4bfae1-2652-4f66-9fc4-078cb4b2facc None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:21.228900 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4bfae1-2652-4f66-9fc4-078cb4b2facc None admin] PATH_INFO: `/v3/projects/659ee9038adf47e585b94881867c18c2/groups/5c1936c14be64aa491d923cbd86f3988/roles/a0fae926490542b4977d1ecf27fcece4` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:21.231077 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4bfae1-2652-4f66-9fc4-078cb4b2facc None admin] RBAC: Authorizing `identity:create_grant(project_id=659ee9038adf47e585b94881867c18c2, group_id=5c1936c14be64aa491d923cbd86f3988, role_id=a0fae926490542b4977d1ecf27fcece4)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:21.232519 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4bfae1-2652-4f66-9fc4-078cb4b2facc None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:21.286126 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 202/398] 10.222.0.42 () {58 vars in 1442 bytes} [Thu Sep 15 07:20:21 2022] PUT /identity/v3/projects/659ee9038adf47e585b94881867c18c2/groups/5c1936c14be64aa491d923cbd86f3988/roles/a0fae926490542b4977d1ecf27fcece4 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:20:21.287462 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e83c399-7cec-4768-a9e7-3ad7f5be4e1b None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:21.287871 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e83c399-7cec-4768-a9e7-3ad7f5be4e1b None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:21.288147 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e83c399-7cec-4768-a9e7-3ad7f5be4e1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:21.309735 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3cc430-965c-4de1-8444-e851c5b5ed86 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:21.317006 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-7e83c399-7cec-4768-a9e7-3ad7f5be4e1b None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:20:21.320608 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-7e83c399-7cec-4768-a9e7-3ad7f5be4e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:20:21.359743 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05475dd7-6aea-4e8e-8210-1957a48624f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:21.360894 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05475dd7-6aea-4e8e-8210-1957a48624f7 None admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:21.361514 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05475dd7-6aea-4e8e-8210-1957a48624f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:21.362070 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05475dd7-6aea-4e8e-8210-1957a48624f7 None admin] PATH_INFO: `/v3/projects/659ee9038adf47e585b94881867c18c2/groups/5c1936c14be64aa491d923cbd86f3988/roles/053e12af84e943e699460a9d76c2072b` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:21.364511 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05475dd7-6aea-4e8e-8210-1957a48624f7 None admin] RBAC: Authorizing `identity:create_grant(project_id=659ee9038adf47e585b94881867c18c2, group_id=5c1936c14be64aa491d923cbd86f3988, role_id=053e12af84e943e699460a9d76c2072b)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:21.366620 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05475dd7-6aea-4e8e-8210-1957a48624f7 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:21.409988 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 203/399] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:20:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 07:20:21.411219 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e83c399-7cec-4768-a9e7-3ad7f5be4e1b None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:21.424216 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 197/400] 10.222.0.42 () {58 vars in 1442 bytes} [Thu Sep 15 07:20:21 2022] PUT /identity/v3/projects/659ee9038adf47e585b94881867c18c2/groups/5c1936c14be64aa491d923cbd86f3988/roles/053e12af84e943e699460a9d76c2072b => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:20:21.451020 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcdbe45-2b9a-42af-9b24-68d51d1426d7 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:21.451898 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcdbe45-2b9a-42af-9b24-68d51d1426d7 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:21.452167 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcdbe45-2b9a-42af-9b24-68d51d1426d7 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:21.458247 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 198/401] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:20:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 07:20:21.460260 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdc653c-5ea5-4d4c-91d7-9f3ae4d74164 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:21.460546 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdc653c-5ea5-4d4c-91d7-9f3ae4d74164 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:21.460820 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdc653c-5ea5-4d4c-91d7-9f3ae4d74164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:21.484074 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-bcdc653c-5ea5-4d4c-91d7-9f3ae4d74164 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:20:21.485335 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-bcdc653c-5ea5-4d4c-91d7-9f3ae4d74164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:20:21.503189 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:21.548968 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:21.603245 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 199/402] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:20:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 07:20:21.615947 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e83c399-7cec-4768-a9e7-3ad7f5be4e1b None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:21.672816 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d77b7c-02d0-415e-8d94-1176e1a2bb5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:21.673459 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d77b7c-02d0-415e-8d94-1176e1a2bb5f None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:21.673785 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d77b7c-02d0-415e-8d94-1176e1a2bb5f None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:21.674124 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d77b7c-02d0-415e-8d94-1176e1a2bb5f None admin] PATH_INFO: `/v3/groups/5c1936c14be64aa491d923cbd86f3988` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:21.675219 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d77b7c-02d0-415e-8d94-1176e1a2bb5f None admin] RBAC: Authorizing `identity:get_group(group_id=5c1936c14be64aa491d923cbd86f3988)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:21.676644 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d77b7c-02d0-415e-8d94-1176e1a2bb5f None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:21.678237 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 204/403] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Sep 15 07:20:21 2022] GET /identity/v3/groups/5c1936c14be64aa491d923cbd86f3988 => generated 226 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:21.684103 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdc653c-5ea5-4d4c-91d7-9f3ae4d74164 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:21.790531 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdc653c-5ea5-4d4c-91d7-9f3ae4d74164 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:21.810622 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d77b7c-02d0-415e-8d94-1176e1a2bb5f None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:21.835718 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4560a2-672f-429e-8b86-b62a71506a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:21.836366 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4560a2-672f-429e-8b86-b62a71506a99 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:21.836641 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4560a2-672f-429e-8b86-b62a71506a99 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:21.836910 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4560a2-672f-429e-8b86-b62a71506a99 None admin] PATH_INFO: `/v3/projects/91757d436677474ab5f1027263238244` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:21.840668 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4560a2-672f-429e-8b86-b62a71506a99 None admin] RBAC: Authorizing `identity:get_project(project_id=91757d436677474ab5f1027263238244)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:21.842750 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4560a2-672f-429e-8b86-b62a71506a99 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:21.844409 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 200/404] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 15 07:20:21 2022] GET /identity/v3/projects/91757d436677474ab5f1027263238244 => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:21.853414 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4560a2-672f-429e-8b86-b62a71506a99 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:21.891830 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:21.952752 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d77b7c-02d0-415e-8d94-1176e1a2bb5f None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:21.999252 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35052196-87a0-4007-bf74-450141fd6cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:21.999916 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35052196-87a0-4007-bf74-450141fd6cb6 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:22.000193 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35052196-87a0-4007-bf74-450141fd6cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:22.000311 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4560a2-672f-429e-8b86-b62a71506a99 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:22.000474 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35052196-87a0-4007-bf74-450141fd6cb6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:22.008503 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:22.014193 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35052196-87a0-4007-bf74-450141fd6cb6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:20:22.020522 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-35052196-87a0-4007-bf74-450141fd6cb6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 07:20:22.020522 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:22.020522 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:22.020522 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:22.020522 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:22.020522 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:22.020522 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:22.020522 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:22.020522 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:22.020522 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:22.020522 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:22.020522 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:22.020522 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:20:22.020522 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:20:22.020522 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:20:22.020522 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:20:22.027553 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:20:22.027553 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:20:22.027553 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:20:22.027553 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:20:22.027553 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:22.027553 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:22.027553 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:22.027553 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:22.027553 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:22.027553 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:22.027553 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:22.027553 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:22.027553 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:22.027553 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:22.027553 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:22.027553 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:22.028319 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:22.028319 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:22.028319 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:22.028319 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:22.028319 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:20:22.028319 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:20:22.028319 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:20:22.028319 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:20:22.028319 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:20:22.028319 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:20:22.028319 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 07:20:22.028319 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:20:22.028319 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 205/405] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Sep 15 07:20:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:22.028319 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35052196-87a0-4007-bf74-450141fd6cb6 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:22.058009 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98b9f42-147f-49cb-bb91-9d60bbe3feeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:22.059594 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98b9f42-147f-49cb-bb91-9d60bbe3feeb None admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:22.060331 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98b9f42-147f-49cb-bb91-9d60bbe3feeb None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:22.060935 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98b9f42-147f-49cb-bb91-9d60bbe3feeb None admin] PATH_INFO: `/v3/projects/91757d436677474ab5f1027263238244/groups/5c1936c14be64aa491d923cbd86f3988/roles/053e12af84e943e699460a9d76c2072b` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:22.064550 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98b9f42-147f-49cb-bb91-9d60bbe3feeb None admin] RBAC: Authorizing `identity:create_grant(project_id=91757d436677474ab5f1027263238244, group_id=5c1936c14be64aa491d923cbd86f3988, role_id=053e12af84e943e699460a9d76c2072b)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:22.066273 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98b9f42-147f-49cb-bb91-9d60bbe3feeb None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:22.137459 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35052196-87a0-4007-bf74-450141fd6cb6 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:22.141737 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 201/406] 10.222.0.42 () {58 vars in 1442 bytes} [Thu Sep 15 07:20:21 2022] PUT /identity/v3/projects/91757d436677474ab5f1027263238244/groups/5c1936c14be64aa491d923cbd86f3988/roles/053e12af84e943e699460a9d76c2072b => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:20:22.142992 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98b9f42-147f-49cb-bb91-9d60bbe3feeb None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:22.180127 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464c7061-267f-4b8c-80e8-5fd4bb49fa54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:22.181158 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464c7061-267f-4b8c-80e8-5fd4bb49fa54 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:22.181158 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464c7061-267f-4b8c-80e8-5fd4bb49fa54 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:22.181349 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464c7061-267f-4b8c-80e8-5fd4bb49fa54 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:22.196291 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464c7061-267f-4b8c-80e8-5fd4bb49fa54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:20:22.201788 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-464c7061-267f-4b8c-80e8-5fd4bb49fa54 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 07:20:22.201788 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:22.201788 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:22.201788 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:22.201788 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:22.201788 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:22.201788 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:22.201788 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:22.201788 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:22.201788 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:22.201788 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:22.201788 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:22.201788 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:20:22.201788 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:20:22.201788 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:20:22.201788 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:20:22.202909 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:20:22.202909 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:20:22.202909 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:20:22.202909 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:20:22.202909 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:22.202909 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:22.202909 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:22.202909 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:22.202909 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:22.202909 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:22.202909 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:22.202909 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:22.202909 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:22.202909 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:22.202909 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:22.202909 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:22.203875 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:22.203875 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:22.203875 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:22.203875 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:22.203875 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:20:22.203875 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:20:22.203875 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:20:22.203875 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:20:22.203875 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:20:22.203875 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:20:22.203875 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 07:20:22.203875 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:20:22.203875 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 206/407] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 15 07:20:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:22.209055 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464c7061-267f-4b8c-80e8-5fd4bb49fa54 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:22.252162 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:22.282783 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98b9f42-147f-49cb-bb91-9d60bbe3feeb None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:22.333277 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cda9d66-d195-4e7b-9678-deda5ad49ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:22.334321 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cda9d66-d195-4e7b-9678-deda5ad49ab1 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:22.334808 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cda9d66-d195-4e7b-9678-deda5ad49ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:22.335287 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cda9d66-d195-4e7b-9678-deda5ad49ab1 None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:22.336223 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cda9d66-d195-4e7b-9678-deda5ad49ab1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:22.336735 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cda9d66-d195-4e7b-9678-deda5ad49ab1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:22.338374 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cda9d66-d195-4e7b-9678-deda5ad49ab1 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:22.353776 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 202/408] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 15 07:20:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:22.355609 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cda9d66-d195-4e7b-9678-deda5ad49ab1 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:22.398055 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:22.459668 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464c7061-267f-4b8c-80e8-5fd4bb49fa54 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:22.479401 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cda9d66-d195-4e7b-9678-deda5ad49ab1 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:22.504434 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6b15c6-2996-4a11-8847-f43bcdae531c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:22.505057 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6b15c6-2996-4a11-8847-f43bcdae531c None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:22.505332 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6b15c6-2996-4a11-8847-f43bcdae531c None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:22.505632 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6b15c6-2996-4a11-8847-f43bcdae531c None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:22.506319 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6b15c6-2996-4a11-8847-f43bcdae531c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:22.506620 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6b15c6-2996-4a11-8847-f43bcdae531c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:22.508054 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6b15c6-2996-4a11-8847-f43bcdae531c None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:22.518651 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 207/409] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 15 07:20:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:22.524233 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6b15c6-2996-4a11-8847-f43bcdae531c None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:22.531171 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef0f34d-d525-4bb0-a20a-2a0a3817aafe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:22.532118 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef0f34d-d525-4bb0-a20a-2a0a3817aafe None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:22.532606 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef0f34d-d525-4bb0-a20a-2a0a3817aafe None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:22.533181 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef0f34d-d525-4bb0-a20a-2a0a3817aafe None admin] PATH_INFO: `/v3/groups/5c1936c14be64aa491d923cbd86f3988` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:22.534682 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef0f34d-d525-4bb0-a20a-2a0a3817aafe None admin] RBAC: Authorizing `identity:get_group(group_id=5c1936c14be64aa491d923cbd86f3988)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:22.536457 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef0f34d-d525-4bb0-a20a-2a0a3817aafe None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:22.538332 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 203/410] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Sep 15 07:20:22 2022] GET /identity/v3/groups/5c1936c14be64aa491d923cbd86f3988 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:20:22.545061 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef0f34d-d525-4bb0-a20a-2a0a3817aafe None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:22.555852 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:22.684581 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6b15c6-2996-4a11-8847-f43bcdae531c None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:22.707488 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef0f34d-d525-4bb0-a20a-2a0a3817aafe None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:22.719756 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:22.735100 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51adf45-940a-45f0-8f93-d09ade2432af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:22.736092 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51adf45-940a-45f0-8f93-d09ade2432af None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:22.736590 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51adf45-940a-45f0-8f93-d09ade2432af None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:22.737095 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51adf45-940a-45f0-8f93-d09ade2432af None admin] PATH_INFO: `/v3/groups/d47a44a719af4dcd80064b386ac52a77` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:22.738257 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51adf45-940a-45f0-8f93-d09ade2432af None admin] RBAC: Authorizing `identity:get_group(group_id=d47a44a719af4dcd80064b386ac52a77)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:22.739710 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51adf45-940a-45f0-8f93-d09ade2432af None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:22.741300 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 208/411] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Sep 15 07:20:22 2022] GET /identity/v3/groups/d47a44a719af4dcd80064b386ac52a77 => generated 229 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:20:22.747875 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51adf45-940a-45f0-8f93-d09ade2432af None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:22.767710 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69468e9f-1cea-4d9c-a2d8-1abbc4697451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:22.768364 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69468e9f-1cea-4d9c-a2d8-1abbc4697451 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:22.768641 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69468e9f-1cea-4d9c-a2d8-1abbc4697451 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:22.768910 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69468e9f-1cea-4d9c-a2d8-1abbc4697451 None admin] PATH_INFO: `/v3/projects/91757d436677474ab5f1027263238244` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:22.771981 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69468e9f-1cea-4d9c-a2d8-1abbc4697451 None admin] RBAC: Authorizing `identity:get_project(project_id=91757d436677474ab5f1027263238244)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:22.773408 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69468e9f-1cea-4d9c-a2d8-1abbc4697451 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:22.775219 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 204/412] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 15 07:20:22 2022] GET /identity/v3/projects/91757d436677474ab5f1027263238244 => generated 302 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:20:22.783569 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69468e9f-1cea-4d9c-a2d8-1abbc4697451 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:22.887222 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:22.915225 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69468e9f-1cea-4d9c-a2d8-1abbc4697451 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:22.957272 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51adf45-940a-45f0-8f93-d09ade2432af None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:22.957517 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2cd3f7-921b-46d9-a4a7-4aa54280f645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:22.958264 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2cd3f7-921b-46d9-a4a7-4aa54280f645 None admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:22.958553 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2cd3f7-921b-46d9-a4a7-4aa54280f645 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:22.958840 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2cd3f7-921b-46d9-a4a7-4aa54280f645 None admin] PATH_INFO: `/v3/projects/91757d436677474ab5f1027263238244/groups/5c1936c14be64aa491d923cbd86f3988/roles/a0fae926490542b4977d1ecf27fcece4` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:22.961359 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2cd3f7-921b-46d9-a4a7-4aa54280f645 None admin] RBAC: Authorizing `identity:create_grant(project_id=91757d436677474ab5f1027263238244, group_id=5c1936c14be64aa491d923cbd86f3988, role_id=a0fae926490542b4977d1ecf27fcece4)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:22.962844 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2cd3f7-921b-46d9-a4a7-4aa54280f645 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:23.004033 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c470670-dd2b-4603-9f2d-c09512f8de9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:23.004743 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c470670-dd2b-4603-9f2d-c09512f8de9d None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:23.005031 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c470670-dd2b-4603-9f2d-c09512f8de9d None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:23.005309 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c470670-dd2b-4603-9f2d-c09512f8de9d None admin] PATH_INFO: `/v3/projects/918580af0a81452f8da95d60256aaea0` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:23.006526 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c470670-dd2b-4603-9f2d-c09512f8de9d None admin] RBAC: Authorizing `identity:get_project(project_id=918580af0a81452f8da95d60256aaea0)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:23.008032 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c470670-dd2b-4603-9f2d-c09512f8de9d None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:23.013552 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 209/413] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 15 07:20:22 2022] GET /identity/v3/projects/918580af0a81452f8da95d60256aaea0 => generated 344 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:20:23.016216 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c470670-dd2b-4603-9f2d-c09512f8de9d None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:23.035082 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 205/414] 10.222.0.42 () {58 vars in 1442 bytes} [Thu Sep 15 07:20:22 2022] PUT /identity/v3/projects/91757d436677474ab5f1027263238244/groups/5c1936c14be64aa491d923cbd86f3988/roles/a0fae926490542b4977d1ecf27fcece4 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 07:20:23.090510 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:23.142437 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c470670-dd2b-4603-9f2d-c09512f8de9d None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:23.182070 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf3eacf-43a8-46c9-bab7-699d0ac88269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:23.182825 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf3eacf-43a8-46c9-bab7-699d0ac88269 None admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:23.183113 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf3eacf-43a8-46c9-bab7-699d0ac88269 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:23.183393 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf3eacf-43a8-46c9-bab7-699d0ac88269 None admin] PATH_INFO: `/v3/projects/918580af0a81452f8da95d60256aaea0/groups/d47a44a719af4dcd80064b386ac52a77/roles/6bbb643e0d8d4bc0a9b27c26edf59216` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:23.185491 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf3eacf-43a8-46c9-bab7-699d0ac88269 None admin] RBAC: Authorizing `identity:create_grant(project_id=918580af0a81452f8da95d60256aaea0, group_id=d47a44a719af4dcd80064b386ac52a77, role_id=6bbb643e0d8d4bc0a9b27c26edf59216)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:23.187094 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf3eacf-43a8-46c9-bab7-699d0ac88269 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:23.266217 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 210/415] 10.222.0.42 () {58 vars in 1442 bytes} [Thu Sep 15 07:20:23 2022] PUT /identity/v3/projects/918580af0a81452f8da95d60256aaea0/groups/d47a44a719af4dcd80064b386ac52a77/roles/6bbb643e0d8d4bc0a9b27c26edf59216 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 07:20:24.021568 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f33b545-2594-462f-ad85-163cc2423580 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:24.022096 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f33b545-2594-462f-ad85-163cc2423580 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:24.022201 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f33b545-2594-462f-ad85-163cc2423580 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:24.023836 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 206/416] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:20:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:20:24.030156 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de4576e-f336-49b7-8159-24e0c70d8de5 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:24.030660 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de4576e-f336-49b7-8159-24e0c70d8de5 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:24.030731 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de4576e-f336-49b7-8159-24e0c70d8de5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:24.051104 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-0de4576e-f336-49b7-8159-24e0c70d8de5 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:20:24.052279 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-0de4576e-f336-49b7-8159-24e0c70d8de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:20:24.139302 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 211/417] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:20:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 07:20:24.203823 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93b1205-b1ba-42b9-aa36-14985f03ef80 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:24.204116 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93b1205-b1ba-42b9-aa36-14985f03ef80 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:24.204289 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93b1205-b1ba-42b9-aa36-14985f03ef80 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:24.205943 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 207/418] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:20:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:20:24.211685 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76fc1d8-5ff3-4ecc-8353-c0ad689c0b4d None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:24.212183 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76fc1d8-5ff3-4ecc-8353-c0ad689c0b4d None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:24.212617 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76fc1d8-5ff3-4ecc-8353-c0ad689c0b4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:24.242570 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-b76fc1d8-5ff3-4ecc-8353-c0ad689c0b4d None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:20:24.244132 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-b76fc1d8-5ff3-4ecc-8353-c0ad689c0b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:20:24.304811 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:24.335253 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 212/419] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:20:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 07:20:24.446252 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93b1205-b1ba-42b9-aa36-14985f03ef80 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:24.556990 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93b1205-b1ba-42b9-aa36-14985f03ef80 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:24.593559 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d35fc1-3ccb-40dd-823b-2954903d69b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:24.594224 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d35fc1-3ccb-40dd-823b-2954903d69b2 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:24.594499 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d35fc1-3ccb-40dd-823b-2954903d69b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:24.594772 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d35fc1-3ccb-40dd-823b-2954903d69b2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:24.596669 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:24.608266 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d35fc1-3ccb-40dd-823b-2954903d69b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:20:24.613952 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-91d35fc1-3ccb-40dd-823b-2954903d69b2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 07:20:24.613952 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:24.613952 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:24.613952 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:24.613952 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:24.613952 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:24.613952 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:24.613952 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:24.613952 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:24.613952 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:24.613952 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:24.613952 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:24.613952 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:20:24.613952 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:20:24.613952 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:20:24.613952 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:20:24.614810 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:20:24.614810 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:20:24.614810 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:20:24.614810 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:20:24.614810 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:24.614810 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:24.614810 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:24.614810 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:24.614810 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:24.614810 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:24.614810 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:24.614810 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:24.614810 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:24.614810 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:24.614810 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:24.614810 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:24.615666 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:24.615666 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:24.615666 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:24.615666 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:24.615666 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:20:24.615666 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:20:24.615666 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:20:24.615666 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:20:24.615666 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:20:24.615666 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:20:24.615666 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 07:20:24.615666 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:20:24.615666 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 208/420] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Sep 15 07:20:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:24.621509 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76fc1d8-5ff3-4ecc-8353-c0ad689c0b4d None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:24.747313 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76fc1d8-5ff3-4ecc-8353-c0ad689c0b4d None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:24.784179 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00576664-18fd-43f1-ad80-c3419d0d121f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:24.785210 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00576664-18fd-43f1-ad80-c3419d0d121f None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:24.785738 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00576664-18fd-43f1-ad80-c3419d0d121f None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:24.786222 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00576664-18fd-43f1-ad80-c3419d0d121f None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:24.787122 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00576664-18fd-43f1-ad80-c3419d0d121f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:24.787632 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00576664-18fd-43f1-ad80-c3419d0d121f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:24.789283 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00576664-18fd-43f1-ad80-c3419d0d121f None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:24.799885 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 213/421] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 15 07:20:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:24.805191 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d35fc1-3ccb-40dd-823b-2954903d69b2 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:24.824398 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00576664-18fd-43f1-ad80-c3419d0d121f None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:24.873246 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:24.959072 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d35fc1-3ccb-40dd-823b-2954903d69b2 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:25.005543 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00576664-18fd-43f1-ad80-c3419d0d121f None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:25.028338 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e80042-e6ea-4c7a-865b-74d0a99f947d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:25.028944 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e80042-e6ea-4c7a-865b-74d0a99f947d None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:25.029224 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e80042-e6ea-4c7a-865b-74d0a99f947d None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:25.029492 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e80042-e6ea-4c7a-865b-74d0a99f947d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:25.070047 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108cf2bc-72f4-4726-a0a0-c6b324986fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:25.070753 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108cf2bc-72f4-4726-a0a0-c6b324986fda None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:25.070955 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108cf2bc-72f4-4726-a0a0-c6b324986fda None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:25.071224 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108cf2bc-72f4-4726-a0a0-c6b324986fda None admin] PATH_INFO: `/v3/roles/member` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:25.087333 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108cf2bc-72f4-4726-a0a0-c6b324986fda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:20:25.093324 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-108cf2bc-72f4-4726-a0a0-c6b324986fda None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 07:20:25.093324 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:25.093324 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:25.093324 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:25.093324 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:25.093324 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:25.093324 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:25.093324 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:25.093324 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:25.093324 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:25.093324 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:25.093324 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:25.093324 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:20:25.093324 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:20:25.093324 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:20:25.093324 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:20:25.094253 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:20:25.094253 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:20:25.094253 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:20:25.094253 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:20:25.094253 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:25.094253 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:25.094253 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:25.094253 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:25.094253 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:25.094253 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:25.094253 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:25.094253 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:25.094253 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:25.094253 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:25.094253 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:25.094253 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:25.095043 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:25.095043 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:25.095043 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:25.095043 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:25.095043 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:20:25.095043 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:20:25.095043 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:20:25.095043 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:20:25.095043 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:20:25.095043 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:20:25.095043 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 07:20:25.095043 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:20:25.095043 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 214/422] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 15 07:20:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:25.096172 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663ec00e-2708-475f-9250-fc6f7bb3f7bc None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:25.096360 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663ec00e-2708-475f-9250-fc6f7bb3f7bc None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:25.098166 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663ec00e-2708-475f-9250-fc6f7bb3f7bc None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:25.102197 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 215/423] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:20:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:20:25.103011 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663ec00e-2708-475f-9250-fc6f7bb3f7bc None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:25.121133 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:25.128745 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e80042-e6ea-4c7a-865b-74d0a99f947d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:25.130237 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e80042-e6ea-4c7a-865b-74d0a99f947d None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:25.245185 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 209/424] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 15 07:20:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 07:20:25.246634 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7bde8f-71c2-48d1-95f3-d0ef2faf5c75 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:25.246925 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7bde8f-71c2-48d1-95f3-d0ef2faf5c75 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:25.247197 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7bde8f-71c2-48d1-95f3-d0ef2faf5c75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:25.251502 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:25.257918 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663ec00e-2708-475f-9250-fc6f7bb3f7bc None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:25.267096 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-de7bde8f-71c2-48d1-95f3-d0ef2faf5c75 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:20:25.268296 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-de7bde8f-71c2-48d1-95f3-d0ef2faf5c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:20:25.305044 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473e65a4-8041-4419-9014-885dd577b160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:25.305963 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473e65a4-8041-4419-9014-885dd577b160 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:25.306391 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473e65a4-8041-4419-9014-885dd577b160 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:25.307086 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473e65a4-8041-4419-9014-885dd577b160 None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:25.309119 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473e65a4-8041-4419-9014-885dd577b160 None admin] RBAC: Adding query filter params (name=member) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:25.309641 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473e65a4-8041-4419-9014-885dd577b160 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:25.311815 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473e65a4-8041-4419-9014-885dd577b160 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:25.324527 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 216/425] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Sep 15 07:20:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:25.325853 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473e65a4-8041-4419-9014-885dd577b160 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:25.346565 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 210/426] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:20:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 07:20:25.353986 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7bde8f-71c2-48d1-95f3-d0ef2faf5c75 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:25.393230 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:25.500434 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7bde8f-71c2-48d1-95f3-d0ef2faf5c75 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:25.539762 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473e65a4-8041-4419-9014-885dd577b160 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:25.544986 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a743ddd-c128-44ed-a00e-110b3782f296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:25.545981 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a743ddd-c128-44ed-a00e-110b3782f296 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:25.546551 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a743ddd-c128-44ed-a00e-110b3782f296 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:25.547049 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a743ddd-c128-44ed-a00e-110b3782f296 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:25.601480 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4200ad-51fd-4cbd-b04d-ff9f7da8f133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:25.602487 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4200ad-51fd-4cbd-b04d-ff9f7da8f133 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:25.602973 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4200ad-51fd-4cbd-b04d-ff9f7da8f133 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:25.603471 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4200ad-51fd-4cbd-b04d-ff9f7da8f133 None admin] PATH_INFO: `/v3/projects/659ee9038adf47e585b94881867c18c2` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:25.604802 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4200ad-51fd-4cbd-b04d-ff9f7da8f133 None admin] RBAC: Authorizing `identity:get_project(project_id=659ee9038adf47e585b94881867c18c2)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:25.606474 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4200ad-51fd-4cbd-b04d-ff9f7da8f133 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:25.609902 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 217/427] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 15 07:20:25 2022] GET /identity/v3/projects/659ee9038adf47e585b94881867c18c2 => generated 298 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:25.615056 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4200ad-51fd-4cbd-b04d-ff9f7da8f133 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:25.659392 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a743ddd-c128-44ed-a00e-110b3782f296 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:25.661217 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a743ddd-c128-44ed-a00e-110b3782f296 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:25.690350 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:25.707622 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:25.793124 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4200ad-51fd-4cbd-b04d-ff9f7da8f133 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:25.795836 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 211/428] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 15 07:20:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 07:20:25.797570 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a743ddd-c128-44ed-a00e-110b3782f296 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:25.831283 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3510fa55-524f-4b3b-9401-6467f93f9725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:25.832192 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3510fa55-524f-4b3b-9401-6467f93f9725 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:25.832656 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3510fa55-524f-4b3b-9401-6467f93f9725 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:25.833101 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3510fa55-524f-4b3b-9401-6467f93f9725 None admin] PATH_INFO: `/v3/groups/5c1936c14be64aa491d923cbd86f3988` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:25.834640 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3510fa55-524f-4b3b-9401-6467f93f9725 None admin] RBAC: Authorizing `identity:get_group(group_id=5c1936c14be64aa491d923cbd86f3988)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:25.836266 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3510fa55-524f-4b3b-9401-6467f93f9725 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:25.838583 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 218/429] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Sep 15 07:20:25 2022] GET /identity/v3/groups/5c1936c14be64aa491d923cbd86f3988 => generated 226 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:25.839577 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3510fa55-524f-4b3b-9401-6467f93f9725 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:25.910387 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a743ddd-c128-44ed-a00e-110b3782f296 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:25.935838 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:25.950556 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f95a6c1-e15e-483f-a70b-25b36fa65415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:25.963344 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:25.963470 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f95a6c1-e15e-483f-a70b-25b36fa65415 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:25.963470 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f95a6c1-e15e-483f-a70b-25b36fa65415 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:25.963470 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f95a6c1-e15e-483f-a70b-25b36fa65415 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:25.963699 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3510fa55-524f-4b3b-9401-6467f93f9725 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:25.963780 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f95a6c1-e15e-483f-a70b-25b36fa65415 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:20:25.968783 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-6f95a6c1-e15e-483f-a70b-25b36fa65415 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 07:20:25.968783 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:25.968783 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:25.968783 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:25.968783 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:25.968783 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:25.968783 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:25.968783 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:25.968783 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:25.968783 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:25.968783 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:25.968783 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:25.968783 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:20:25.968783 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:20:25.968783 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:20:25.968783 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:20:25.969698 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:20:25.969698 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:20:25.969698 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:20:25.969698 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:20:25.969698 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:25.969698 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:25.969698 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:25.969698 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:25.969698 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:25.969698 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:25.969698 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:25.969698 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:25.969698 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:25.969698 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:25.969698 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:25.969698 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:25.970598 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:25.970598 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:25.970598 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:25.970598 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:25.970598 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:20:25.970598 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:20:25.970598 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:20:25.970598 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:20:25.970598 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:20:25.970598 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:20:25.970598 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 07:20:25.970598 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:20:25.970598 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 212/430] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 15 07:20:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:25.971265 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f95a6c1-e15e-483f-a70b-25b36fa65415 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:25.997012 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef48bbd-4e6d-4aad-8514-d552f28470c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:25.997690 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef48bbd-4e6d-4aad-8514-d552f28470c2 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:25.997976 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef48bbd-4e6d-4aad-8514-d552f28470c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:25.998252 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef48bbd-4e6d-4aad-8514-d552f28470c2 None admin] PATH_INFO: `/v3/projects/659ee9038adf47e585b94881867c18c2` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:25.999454 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef48bbd-4e6d-4aad-8514-d552f28470c2 None admin] RBAC: Authorizing `identity:get_project(project_id=659ee9038adf47e585b94881867c18c2)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:26.000921 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef48bbd-4e6d-4aad-8514-d552f28470c2 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:26.006980 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 219/431] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 15 07:20:25 2022] GET /identity/v3/projects/659ee9038adf47e585b94881867c18c2 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:26.006980 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac758027-8863-45fc-9765-1de792db1bcc None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:26.006980 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac758027-8863-45fc-9765-1de792db1bcc None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:26.006980 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac758027-8863-45fc-9765-1de792db1bcc None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:26.006980 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 220/432] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:20:26 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:20:26.006980 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac758027-8863-45fc-9765-1de792db1bcc None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:26.095644 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f95a6c1-e15e-483f-a70b-25b36fa65415 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:26.120008 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac758027-8863-45fc-9765-1de792db1bcc None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:26.130687 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ae9a54-b937-4ac0-bd04-6d0640e35571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:26.131305 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ae9a54-b937-4ac0-bd04-6d0640e35571 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:26.131592 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ae9a54-b937-4ac0-bd04-6d0640e35571 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:26.131900 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ae9a54-b937-4ac0-bd04-6d0640e35571 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:26.132629 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ae9a54-b937-4ac0-bd04-6d0640e35571 None admin] RBAC: Adding query filter params (group.id=5c1936c14be64aa491d923cbd86f3988, role.id=a0fae926490542b4977d1ecf27fcece4, scope.project.id=659ee9038adf47e585b94881867c18c2) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:26.132943 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ae9a54-b937-4ac0-bd04-6d0640e35571 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:26.134416 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ae9a54-b937-4ac0-bd04-6d0640e35571 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:26.140743 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 213/433] 10.222.0.42 () {56 vars in 1376 bytes} [Thu Sep 15 07:20:25 2022] GET /identity/v3/role_assignments?group.id=5c1936c14be64aa491d923cbd86f3988&scope.project.id=659ee9038adf47e585b94881867c18c2&role.id=a0fae926490542b4977d1ecf27fcece4 => generated 611 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:26.141545 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ae9a54-b937-4ac0-bd04-6d0640e35571 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:26.177915 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaf72e4-aa80-41b1-b629-fd0eb79cdd3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:26.177915 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaf72e4-aa80-41b1-b629-fd0eb79cdd3b None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:26.177915 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaf72e4-aa80-41b1-b629-fd0eb79cdd3b None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:26.177915 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaf72e4-aa80-41b1-b629-fd0eb79cdd3b None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:26.178283 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaf72e4-aa80-41b1-b629-fd0eb79cdd3b None admin] RBAC: Adding query filter params (name=member) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:26.178283 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaf72e4-aa80-41b1-b629-fd0eb79cdd3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:26.178283 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaf72e4-aa80-41b1-b629-fd0eb79cdd3b None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:26.185808 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 221/434] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Sep 15 07:20:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:26.188088 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a7ff9-6005-4315-ab9c-42a6849b1521 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:26.188551 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a7ff9-6005-4315-ab9c-42a6849b1521 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:26.189023 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a7ff9-6005-4315-ab9c-42a6849b1521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:26.208275 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-760a7ff9-6005-4315-ab9c-42a6849b1521 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:20:26.209517 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-760a7ff9-6005-4315-ab9c-42a6849b1521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:20:26.212183 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:26.233550 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:26.270982 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ae9a54-b937-4ac0-bd04-6d0640e35571 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:26.287145 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 222/435] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:20:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 07:20:26.289055 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02446b1-d66f-4d4b-af12-51dd786588d9 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:26.290635 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02446b1-d66f-4d4b-af12-51dd786588d9 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:26.291054 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02446b1-d66f-4d4b-af12-51dd786588d9 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:26.292499 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 223/436] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:20:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:20:26.293347 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02446b1-d66f-4d4b-af12-51dd786588d9 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:26.338980 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853e71e1-0c1d-4875-b307-0d6ded58d342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:26.339982 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853e71e1-0c1d-4875-b307-0d6ded58d342 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:26.341012 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853e71e1-0c1d-4875-b307-0d6ded58d342 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:26.341328 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853e71e1-0c1d-4875-b307-0d6ded58d342 None admin] PATH_INFO: `/v3/groups/5c1936c14be64aa491d923cbd86f3988` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:26.343598 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853e71e1-0c1d-4875-b307-0d6ded58d342 None admin] RBAC: Authorizing `identity:get_group(group_id=5c1936c14be64aa491d923cbd86f3988)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:26.344797 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853e71e1-0c1d-4875-b307-0d6ded58d342 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:26.352372 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 214/437] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Sep 15 07:20:26 2022] GET /identity/v3/groups/5c1936c14be64aa491d923cbd86f3988 => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:26.352372 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e952ad0-aca4-48c7-ab28-51cb0a667233 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:26.352372 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e952ad0-aca4-48c7-ab28-51cb0a667233 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:26.352372 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e952ad0-aca4-48c7-ab28-51cb0a667233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:26.371080 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-8e952ad0-aca4-48c7-ab28-51cb0a667233 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:20:26.372373 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-8e952ad0-aca4-48c7-ab28-51cb0a667233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:20:26.451013 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 215/438] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:20:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 07:20:26.452328 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e952ad0-aca4-48c7-ab28-51cb0a667233 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:26.458755 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02446b1-d66f-4d4b-af12-51dd786588d9 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:26.504157 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e75c457-7701-44f8-900d-af80ab5fd33d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:26.505419 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:26.506316 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e75c457-7701-44f8-900d-af80ab5fd33d None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:26.506316 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e75c457-7701-44f8-900d-af80ab5fd33d None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:26.506711 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e75c457-7701-44f8-900d-af80ab5fd33d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:26.543887 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:26.564798 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e952ad0-aca4-48c7-ab28-51cb0a667233 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:26.594667 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e75c457-7701-44f8-900d-af80ab5fd33d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:26.596394 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e75c457-7701-44f8-900d-af80ab5fd33d None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:26.599531 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c5fcd7-1970-4485-8bba-dd39115b9099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:26.600139 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c5fcd7-1970-4485-8bba-dd39115b9099 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:26.600418 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c5fcd7-1970-4485-8bba-dd39115b9099 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:26.600690 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c5fcd7-1970-4485-8bba-dd39115b9099 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:26.601406 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c5fcd7-1970-4485-8bba-dd39115b9099 None admin] RBAC: Adding query filter params (group.id=5c1936c14be64aa491d923cbd86f3988, role.id=053e12af84e943e699460a9d76c2072b, scope.project.id=659ee9038adf47e585b94881867c18c2) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:26.601746 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c5fcd7-1970-4485-8bba-dd39115b9099 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:26.603136 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c5fcd7-1970-4485-8bba-dd39115b9099 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:26.609271 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 216/439] 10.222.0.42 () {56 vars in 1376 bytes} [Thu Sep 15 07:20:26 2022] GET /identity/v3/role_assignments?group.id=5c1936c14be64aa491d923cbd86f3988&scope.project.id=659ee9038adf47e585b94881867c18c2&role.id=053e12af84e943e699460a9d76c2072b => generated 611 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:26.610086 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c5fcd7-1970-4485-8bba-dd39115b9099 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:26.731608 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 224/440] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 15 07:20:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 07:20:26.732920 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e75c457-7701-44f8-900d-af80ab5fd33d None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:26.750664 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c5fcd7-1970-4485-8bba-dd39115b9099 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:26.762746 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:26.788559 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98abed0-ffa1-4076-8ca0-8eea898f2424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:26.789351 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98abed0-ffa1-4076-8ca0-8eea898f2424 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:26.789742 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98abed0-ffa1-4076-8ca0-8eea898f2424 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:26.790079 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98abed0-ffa1-4076-8ca0-8eea898f2424 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:26.797996 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:26.803621 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98abed0-ffa1-4076-8ca0-8eea898f2424 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:20:26.809275 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-e98abed0-ffa1-4076-8ca0-8eea898f2424 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 07:20:26.809275 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:26.809275 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:26.809275 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:26.809275 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:26.809275 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:26.809275 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:26.809275 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:26.809275 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:26.809275 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:26.809275 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:26.809275 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:26.809275 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:20:26.809275 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:20:26.809275 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:20:26.809275 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:20:26.814432 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:20:26.814432 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:20:26.814432 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:20:26.814432 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:20:26.814432 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:26.814432 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:26.814432 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:26.814432 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:26.814432 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:26.814432 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:26.814432 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:26.814432 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:26.814432 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:26.814432 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:26.814432 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:26.814432 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:26.816226 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:26.816226 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:26.816226 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:26.816226 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:26.816226 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:20:26.816226 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:20:26.816226 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:20:26.816226 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:20:26.816226 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:20:26.816226 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:20:26.816226 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 07:20:26.816226 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:20:26.816987 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 217/441] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Sep 15 07:20:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:26.817159 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98abed0-ffa1-4076-8ca0-8eea898f2424 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:26.851603 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e75c457-7701-44f8-900d-af80ab5fd33d None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:26.892087 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afad3f5-29ee-4d18-a384-d8b10b43629c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:26.892973 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afad3f5-29ee-4d18-a384-d8b10b43629c None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:26.893254 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afad3f5-29ee-4d18-a384-d8b10b43629c None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:26.893527 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afad3f5-29ee-4d18-a384-d8b10b43629c None admin] PATH_INFO: `/v3/projects/91757d436677474ab5f1027263238244` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:26.894697 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afad3f5-29ee-4d18-a384-d8b10b43629c None admin] RBAC: Authorizing `identity:get_project(project_id=91757d436677474ab5f1027263238244)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:26.896157 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afad3f5-29ee-4d18-a384-d8b10b43629c None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:26.897966 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 225/442] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 15 07:20:26 2022] GET /identity/v3/projects/91757d436677474ab5f1027263238244 => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:26.902631 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afad3f5-29ee-4d18-a384-d8b10b43629c None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:26.947180 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98abed0-ffa1-4076-8ca0-8eea898f2424 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:26.989571 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049e83db-db1f-4707-9af7-5b958d0c4076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:26.990165 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049e83db-db1f-4707-9af7-5b958d0c4076 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:26.990911 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049e83db-db1f-4707-9af7-5b958d0c4076 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:26.990911 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049e83db-db1f-4707-9af7-5b958d0c4076 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:27.004690 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049e83db-db1f-4707-9af7-5b958d0c4076 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:20:27.010292 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-049e83db-db1f-4707-9af7-5b958d0c4076 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 07:20:27.010292 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:27.010292 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:27.010292 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:27.010292 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:27.010292 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:27.010292 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:27.010292 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:27.010292 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:27.010292 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:27.010292 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:27.010292 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:27.010292 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:20:27.010292 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:20:27.010292 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:20:27.010292 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:20:27.011170 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:20:27.011170 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:20:27.011170 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:20:27.011170 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:20:27.011170 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:27.011170 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:27.011170 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:27.011170 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:27.011170 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:27.011170 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:27.011170 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:27.011170 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:27.011170 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:27.011170 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:27.011170 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:27.011170 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:27.012290 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:27.012290 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:27.012290 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:27.012290 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:27.012290 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:20:27.012290 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:20:27.012290 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:20:27.012290 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:20:27.012290 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:20:27.012290 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:20:27.012290 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 07:20:27.012290 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:20:27.012290 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 218/443] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 15 07:20:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:27.012290 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049e83db-db1f-4707-9af7-5b958d0c4076 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:27.031472 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afad3f5-29ee-4d18-a384-d8b10b43629c None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:27.049298 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:27.062367 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:27.077415 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12513e38-e2c4-4c48-8e69-1e49342f2f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:27.078385 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12513e38-e2c4-4c48-8e69-1e49342f2f56 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:27.078765 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12513e38-e2c4-4c48-8e69-1e49342f2f56 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:27.078942 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12513e38-e2c4-4c48-8e69-1e49342f2f56 None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:27.079603 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12513e38-e2c4-4c48-8e69-1e49342f2f56 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:27.079913 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12513e38-e2c4-4c48-8e69-1e49342f2f56 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:27.081293 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12513e38-e2c4-4c48-8e69-1e49342f2f56 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:27.090197 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 226/444] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 15 07:20:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:27.091023 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12513e38-e2c4-4c48-8e69-1e49342f2f56 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:27.133695 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049e83db-db1f-4707-9af7-5b958d0c4076 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:27.177767 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1afccc0-9882-4ba3-b67c-4e3fee5e5873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:27.178482 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1afccc0-9882-4ba3-b67c-4e3fee5e5873 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:27.178702 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1afccc0-9882-4ba3-b67c-4e3fee5e5873 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:27.178976 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1afccc0-9882-4ba3-b67c-4e3fee5e5873 None admin] PATH_INFO: `/v3/groups/5c1936c14be64aa491d923cbd86f3988` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:27.180131 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1afccc0-9882-4ba3-b67c-4e3fee5e5873 None admin] RBAC: Authorizing `identity:get_group(group_id=5c1936c14be64aa491d923cbd86f3988)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:27.181667 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1afccc0-9882-4ba3-b67c-4e3fee5e5873 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:27.183595 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 219/445] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Sep 15 07:20:27 2022] GET /identity/v3/groups/5c1936c14be64aa491d923cbd86f3988 => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:27.184377 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1afccc0-9882-4ba3-b67c-4e3fee5e5873 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:27.212530 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12513e38-e2c4-4c48-8e69-1e49342f2f56 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:27.246376 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc50af45-85bf-4663-9dbe-de8bd4048cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:27.246911 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc50af45-85bf-4663-9dbe-de8bd4048cdc None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:27.247192 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc50af45-85bf-4663-9dbe-de8bd4048cdc None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:27.247475 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc50af45-85bf-4663-9dbe-de8bd4048cdc None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:27.248127 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc50af45-85bf-4663-9dbe-de8bd4048cdc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:27.248449 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc50af45-85bf-4663-9dbe-de8bd4048cdc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:27.249853 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc50af45-85bf-4663-9dbe-de8bd4048cdc None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:27.258284 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 227/446] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 15 07:20:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:27.259077 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc50af45-85bf-4663-9dbe-de8bd4048cdc None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:27.282747 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:27.286383 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1afccc0-9882-4ba3-b67c-4e3fee5e5873 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:27.308613 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:27.320685 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddeb35a-c145-4441-a1bb-0c7b8e7259fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:27.321295 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddeb35a-c145-4441-a1bb-0c7b8e7259fa None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:27.321596 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddeb35a-c145-4441-a1bb-0c7b8e7259fa None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:27.321870 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddeb35a-c145-4441-a1bb-0c7b8e7259fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:27.365429 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc50af45-85bf-4663-9dbe-de8bd4048cdc None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:27.398739 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353bf9b5-13ed-4e2a-a906-c49c603b1feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:27.399342 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353bf9b5-13ed-4e2a-a906-c49c603b1feb None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:27.399623 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353bf9b5-13ed-4e2a-a906-c49c603b1feb None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:27.399894 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353bf9b5-13ed-4e2a-a906-c49c603b1feb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:27.400618 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353bf9b5-13ed-4e2a-a906-c49c603b1feb None admin] RBAC: Adding query filter params (group.id=5c1936c14be64aa491d923cbd86f3988, role.id=053e12af84e943e699460a9d76c2072b, scope.project.id=91757d436677474ab5f1027263238244) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:27.400925 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353bf9b5-13ed-4e2a-a906-c49c603b1feb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:27.402479 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353bf9b5-13ed-4e2a-a906-c49c603b1feb None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:27.417647 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 228/447] 10.222.0.42 () {56 vars in 1376 bytes} [Thu Sep 15 07:20:27 2022] GET /identity/v3/role_assignments?group.id=5c1936c14be64aa491d923cbd86f3988&scope.project.id=91757d436677474ab5f1027263238244&role.id=053e12af84e943e699460a9d76c2072b => generated 611 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:27.417647 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353bf9b5-13ed-4e2a-a906-c49c603b1feb None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:27.417936 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddeb35a-c145-4441-a1bb-0c7b8e7259fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:27.417936 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddeb35a-c145-4441-a1bb-0c7b8e7259fa None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:27.532660 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:27.536192 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353bf9b5-13ed-4e2a-a906-c49c603b1feb None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:27.537711 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 220/448] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 15 07:20:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 07:20:27.543698 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddeb35a-c145-4441-a1bb-0c7b8e7259fa None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:27.552631 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:27.577549 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc857ef2-9ce7-4173-9207-2f6e7e2d09ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:27.578268 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc857ef2-9ce7-4173-9207-2f6e7e2d09ca None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:27.578557 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc857ef2-9ce7-4173-9207-2f6e7e2d09ca None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:27.578833 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc857ef2-9ce7-4173-9207-2f6e7e2d09ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:27.661084 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc857ef2-9ce7-4173-9207-2f6e7e2d09ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:27.662759 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc857ef2-9ce7-4173-9207-2f6e7e2d09ca None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:27.669865 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddeb35a-c145-4441-a1bb-0c7b8e7259fa None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:27.725564 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee960a45-9195-4a26-b982-372476f91a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:27.726253 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee960a45-9195-4a26-b982-372476f91a41 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:27.726533 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee960a45-9195-4a26-b982-372476f91a41 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:27.726807 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee960a45-9195-4a26-b982-372476f91a41 None admin] PATH_INFO: `/v3/projects/91757d436677474ab5f1027263238244` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:27.727933 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee960a45-9195-4a26-b982-372476f91a41 None admin] RBAC: Authorizing `identity:get_project(project_id=91757d436677474ab5f1027263238244)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:27.729366 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee960a45-9195-4a26-b982-372476f91a41 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:27.731340 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 221/449] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 15 07:20:27 2022] GET /identity/v3/projects/91757d436677474ab5f1027263238244 => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:20:27.738459 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee960a45-9195-4a26-b982-372476f91a41 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:27.793910 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 229/450] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 15 07:20:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 07:20:27.800587 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc857ef2-9ce7-4173-9207-2f6e7e2d09ca None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:27.815227 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:27.817754 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:27.850681 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee960a45-9195-4a26-b982-372476f91a41 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:27.885175 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb182f94-55f8-4fc0-94d7-697e8227c8d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:27.885833 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb182f94-55f8-4fc0-94d7-697e8227c8d8 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:27.886116 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb182f94-55f8-4fc0-94d7-697e8227c8d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:27.886390 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb182f94-55f8-4fc0-94d7-697e8227c8d8 None admin] PATH_INFO: `/v3/groups/5c1936c14be64aa491d923cbd86f3988` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:27.887495 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb182f94-55f8-4fc0-94d7-697e8227c8d8 None admin] RBAC: Authorizing `identity:get_group(group_id=5c1936c14be64aa491d923cbd86f3988)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:27.888924 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb182f94-55f8-4fc0-94d7-697e8227c8d8 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:27.891413 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 222/451] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Sep 15 07:20:27 2022] GET /identity/v3/groups/5c1936c14be64aa491d923cbd86f3988 => generated 226 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:20:27.897274 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb182f94-55f8-4fc0-94d7-697e8227c8d8 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:27.903189 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc857ef2-9ce7-4173-9207-2f6e7e2d09ca None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:27.937204 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4ad4e6-22de-4f73-ace3-b73bfc18f033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:27.937863 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4ad4e6-22de-4f73-ace3-b73bfc18f033 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:27.938143 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4ad4e6-22de-4f73-ace3-b73bfc18f033 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:27.938412 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4ad4e6-22de-4f73-ace3-b73bfc18f033 None admin] PATH_INFO: `/v3/projects/918580af0a81452f8da95d60256aaea0` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:27.939553 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4ad4e6-22de-4f73-ace3-b73bfc18f033 None admin] RBAC: Authorizing `identity:get_project(project_id=918580af0a81452f8da95d60256aaea0)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:27.940976 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4ad4e6-22de-4f73-ace3-b73bfc18f033 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:27.942620 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 230/452] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 15 07:20:27 2022] GET /identity/v3/projects/918580af0a81452f8da95d60256aaea0 => generated 344 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:20:27.947957 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4ad4e6-22de-4f73-ace3-b73bfc18f033 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:28.002174 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb182f94-55f8-4fc0-94d7-697e8227c8d8 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:28.032888 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:28.038975 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04dc3d0-fd9a-47f2-9e23-cf84fe039e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:28.039660 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04dc3d0-fd9a-47f2-9e23-cf84fe039e33 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:28.039941 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04dc3d0-fd9a-47f2-9e23-cf84fe039e33 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:28.040207 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04dc3d0-fd9a-47f2-9e23-cf84fe039e33 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:28.040921 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04dc3d0-fd9a-47f2-9e23-cf84fe039e33 None admin] RBAC: Adding query filter params (group.id=5c1936c14be64aa491d923cbd86f3988, role.id=a0fae926490542b4977d1ecf27fcece4, scope.project.id=91757d436677474ab5f1027263238244) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:28.041228 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04dc3d0-fd9a-47f2-9e23-cf84fe039e33 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:28.042871 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04dc3d0-fd9a-47f2-9e23-cf84fe039e33 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:28.044942 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:28.050366 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 223/453] 10.222.0.42 () {56 vars in 1376 bytes} [Thu Sep 15 07:20:27 2022] GET /identity/v3/role_assignments?group.id=5c1936c14be64aa491d923cbd86f3988&scope.project.id=91757d436677474ab5f1027263238244&role.id=a0fae926490542b4977d1ecf27fcece4 => generated 611 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:20:28.052938 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4ad4e6-22de-4f73-ace3-b73bfc18f033 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:28.091734 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9874d5ad-6c4e-46ef-915f-b5007e328185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:28.093109 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9874d5ad-6c4e-46ef-915f-b5007e328185 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:28.093507 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9874d5ad-6c4e-46ef-915f-b5007e328185 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:28.093821 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9874d5ad-6c4e-46ef-915f-b5007e328185 None admin] PATH_INFO: `/v3/groups/d47a44a719af4dcd80064b386ac52a77` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:28.095003 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9874d5ad-6c4e-46ef-915f-b5007e328185 None admin] RBAC: Authorizing `identity:get_group(group_id=d47a44a719af4dcd80064b386ac52a77)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:28.096465 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9874d5ad-6c4e-46ef-915f-b5007e328185 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:28.098112 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 231/454] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Sep 15 07:20:27 2022] GET /identity/v3/groups/d47a44a719af4dcd80064b386ac52a77 => generated 229 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:20:28.104269 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04dc3d0-fd9a-47f2-9e23-cf84fe039e33 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:28.213168 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04dc3d0-fd9a-47f2-9e23-cf84fe039e33 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:28.248048 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e40ad0-378b-4ef3-a396-e0a08c948f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:28.249641 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e40ad0-378b-4ef3-a396-e0a08c948f76 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:28.249928 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e40ad0-378b-4ef3-a396-e0a08c948f76 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:28.250199 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e40ad0-378b-4ef3-a396-e0a08c948f76 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:28.250920 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e40ad0-378b-4ef3-a396-e0a08c948f76 None admin] RBAC: Adding query filter params (group.id=d47a44a719af4dcd80064b386ac52a77, role.id=6bbb643e0d8d4bc0a9b27c26edf59216, scope.project.id=918580af0a81452f8da95d60256aaea0) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:28.251228 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e40ad0-378b-4ef3-a396-e0a08c948f76 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:28.252647 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e40ad0-378b-4ef3-a396-e0a08c948f76 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:28.258498 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 224/455] 10.222.0.42 () {56 vars in 1376 bytes} [Thu Sep 15 07:20:28 2022] GET /identity/v3/role_assignments?group.id=d47a44a719af4dcd80064b386ac52a77&scope.project.id=918580af0a81452f8da95d60256aaea0&role.id=6bbb643e0d8d4bc0a9b27c26edf59216 => generated 611 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:20:31.005675 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f423aee-ebe6-410a-9786-74b15450ff3a None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:31.006215 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f423aee-ebe6-410a-9786-74b15450ff3a None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:31.006215 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f423aee-ebe6-410a-9786-74b15450ff3a None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:31.007917 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 232/456] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:20:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:20:31.013944 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdc05d3-1db9-4663-a597-11ebd9553cc1 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:31.014550 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdc05d3-1db9-4663-a597-11ebd9553cc1 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:31.014550 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdc05d3-1db9-4663-a597-11ebd9553cc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:31.042805 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-1bdc05d3-1db9-4663-a597-11ebd9553cc1 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:20:31.044052 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-1bdc05d3-1db9-4663-a597-11ebd9553cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:20:31.124020 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:31.140465 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 225/457] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:20:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 07:20:31.178823 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72f5843-91f6-4b2f-92fc-efaf33986039 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:31.179099 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72f5843-91f6-4b2f-92fc-efaf33986039 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:31.179380 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72f5843-91f6-4b2f-92fc-efaf33986039 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:31.180902 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 233/458] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:20:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:20:31.186826 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708b04a3-a07b-4dfa-8d99-3400919afc6a None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:31.187122 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708b04a3-a07b-4dfa-8d99-3400919afc6a None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:31.187398 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708b04a3-a07b-4dfa-8d99-3400919afc6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:31.211966 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-708b04a3-a07b-4dfa-8d99-3400919afc6a None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:20:31.211966 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-708b04a3-a07b-4dfa-8d99-3400919afc6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:20:31.215069 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b81b717-931e-46b2-94c3-adf52099a4bc None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:31.215359 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b81b717-931e-46b2-94c3-adf52099a4bc None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:31.215638 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b81b717-931e-46b2-94c3-adf52099a4bc None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:31.217143 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 234/459] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:20:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 07:20:31.222704 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8783a3-0aa2-43fd-a4ec-a39b5e833e60 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:31.222989 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8783a3-0aa2-43fd-a4ec-a39b5e833e60 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:31.223267 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8783a3-0aa2-43fd-a4ec-a39b5e833e60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:31.253121 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-ff8783a3-0aa2-43fd-a4ec-a39b5e833e60 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:20:31.254337 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-ff8783a3-0aa2-43fd-a4ec-a39b5e833e60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:20:31.334968 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 226/460] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:20:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 07:20:31.364306 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 235/461] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:20:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 07:20:31.548292 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708b04a3-a07b-4dfa-8d99-3400919afc6a None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:31.668582 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8783a3-0aa2-43fd-a4ec-a39b5e833e60 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:31.701166 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708b04a3-a07b-4dfa-8d99-3400919afc6a None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:31.737639 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:31.751572 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c8351d-5e9a-4a21-8fcb-80f547646894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:31.752217 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c8351d-5e9a-4a21-8fcb-80f547646894 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:31.753250 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c8351d-5e9a-4a21-8fcb-80f547646894 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:31.753250 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c8351d-5e9a-4a21-8fcb-80f547646894 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:31.761405 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:31.764287 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c8351d-5e9a-4a21-8fcb-80f547646894 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:31.766056 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c8351d-5e9a-4a21-8fcb-80f547646894 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:31.782166 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-a5c8351d-5e9a-4a21-8fcb-80f547646894 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:20:31.782166 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:31.782166 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:31.782166 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:31.782166 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:31.782166 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:31.782166 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:31.782166 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:31.782166 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:31.782166 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:31.782166 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:31.782166 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:31.782166 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 07:20:31.782166 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 07:20:31.782166 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 07:20:31.782166 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 07:20:31.783091 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:31.783091 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:31.783091 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:31.783091 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:31.783091 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:31.783091 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:31.783091 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:31.783091 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:31.783091 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:31.783091 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:31.783091 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:31.783091 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:31.783091 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:31.783091 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:31.783091 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:31.783091 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:31.783091 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 07:20:31.783928 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 07:20:31.783928 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:20:31.783928 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:20:31.783928 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 227/462] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:20:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:31.784577 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c8351d-5e9a-4a21-8fcb-80f547646894 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:31.790098 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8783a3-0aa2-43fd-a4ec-a39b5e833e60 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:31.828774 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b6334d-f9cf-4b68-91de-332c061aaca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:31.829328 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b6334d-f9cf-4b68-91de-332c061aaca9 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:31.829633 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b6334d-f9cf-4b68-91de-332c061aaca9 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:31.829915 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b6334d-f9cf-4b68-91de-332c061aaca9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:31.840092 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b6334d-f9cf-4b68-91de-332c061aaca9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:31.841547 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b6334d-f9cf-4b68-91de-332c061aaca9 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:31.857406 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-f1b6334d-f9cf-4b68-91de-332c061aaca9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:20:31.857406 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:31.857406 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:31.857406 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:31.857406 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:31.857406 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:31.857406 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:31.857406 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:31.857406 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:31.857406 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:31.857406 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:31.857406 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:31.857406 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 07:20:31.857406 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 07:20:31.857406 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 07:20:31.857406 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 07:20:31.858938 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:31.858938 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:31.858938 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:31.858938 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:31.858938 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:31.858938 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:31.858938 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:31.858938 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:31.858938 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:31.858938 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:31.858938 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:31.858938 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:31.858938 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:31.858938 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:31.858938 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:31.858938 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:31.858938 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 07:20:31.859969 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 07:20:31.859969 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:20:31.859969 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:20:31.864733 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 236/463] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:20:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:31.865915 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5114c37-096a-456b-8319-471ef41d8ee3 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:31.866199 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5114c37-096a-456b-8319-471ef41d8ee3 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:31.866477 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5114c37-096a-456b-8319-471ef41d8ee3 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:31.868255 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 237/464] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:20:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:20:31.868437 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5114c37-096a-456b-8319-471ef41d8ee3 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:31.899939 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c8351d-5e9a-4a21-8fcb-80f547646894 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:31.938699 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d75843-da56-48b8-a9ce-43e4c0fd889f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:31.939711 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d75843-da56-48b8-a9ce-43e4c0fd889f None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:31.940197 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d75843-da56-48b8-a9ce-43e4c0fd889f None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:31.940671 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d75843-da56-48b8-a9ce-43e4c0fd889f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:31.956873 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d75843-da56-48b8-a9ce-43e4c0fd889f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:31.958363 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d75843-da56-48b8-a9ce-43e4c0fd889f None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:31.973430 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-38d75843-da56-48b8-a9ce-43e4c0fd889f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:20:31.973430 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:31.973430 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:31.973430 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:31.973430 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:31.973430 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:31.973430 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:31.973430 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:31.973430 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:31.973430 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:31.973430 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:31.973430 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:31.973430 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 07:20:31.973430 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 07:20:31.973430 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 07:20:31.973430 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 07:20:31.974388 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:31.974388 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:31.974388 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:31.974388 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:31.974388 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:31.974388 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:31.974388 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:31.974388 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:31.974388 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:31.974388 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:31.974388 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:31.974388 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:31.974388 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:31.974388 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:31.974388 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:31.974388 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:31.974388 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 07:20:31.975217 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 07:20:31.975217 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:20:31.975217 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:20:31.975217 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 228/465] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:20:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:31.975431 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d75843-da56-48b8-a9ce-43e4c0fd889f None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:31.988747 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5114c37-096a-456b-8319-471ef41d8ee3 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:32.006049 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:32.020888 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:32.027779 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a42852-9d84-4ae6-9163-3fcfa127d836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:32.028399 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a42852-9d84-4ae6-9163-3fcfa127d836 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:32.028679 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a42852-9d84-4ae6-9163-3fcfa127d836 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:32.028949 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a42852-9d84-4ae6-9163-3fcfa127d836 None admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:32.029600 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a42852-9d84-4ae6-9163-3fcfa127d836 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:32.029985 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a42852-9d84-4ae6-9163-3fcfa127d836 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:32.031725 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a42852-9d84-4ae6-9163-3fcfa127d836 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:32.053907 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 238/466] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 15 07:20:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:32.055563 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f4686d-878d-4f28-97f6-f0fd6429ce22 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:32.055851 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f4686d-878d-4f28-97f6-f0fd6429ce22 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:32.056131 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f4686d-878d-4f28-97f6-f0fd6429ce22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:32.075087 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-e4f4686d-878d-4f28-97f6-f0fd6429ce22 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:20:32.076271 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-e4f4686d-878d-4f28-97f6-f0fd6429ce22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:20:32.085940 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d75843-da56-48b8-a9ce-43e4c0fd889f None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:32.123094 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67ccb52-bcbe-4986-8fa0-be77b66fefa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:32.123804 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67ccb52-bcbe-4986-8fa0-be77b66fefa2 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:32.124127 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67ccb52-bcbe-4986-8fa0-be77b66fefa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:32.124430 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67ccb52-bcbe-4986-8fa0-be77b66fefa2 None admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:32.125171 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67ccb52-bcbe-4986-8fa0-be77b66fefa2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:32.125549 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67ccb52-bcbe-4986-8fa0-be77b66fefa2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:32.126900 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67ccb52-bcbe-4986-8fa0-be77b66fefa2 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:32.139056 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 229/467] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 15 07:20:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:32.144269 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67ccb52-bcbe-4986-8fa0-be77b66fefa2 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:32.150861 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 239/468] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:20:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 07:20:32.162503 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f4686d-878d-4f28-97f6-f0fd6429ce22 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:32.261830 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:32.275762 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f4686d-878d-4f28-97f6-f0fd6429ce22 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:32.283627 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67ccb52-bcbe-4986-8fa0-be77b66fefa2 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:32.300249 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:32.323782 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2e5816-e1ca-45ca-ad1a-b0a1d82a68a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:32.325528 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2e5816-e1ca-45ca-ad1a-b0a1d82a68a7 None admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:32.326466 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2e5816-e1ca-45ca-ad1a-b0a1d82a68a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:32.327210 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2e5816-e1ca-45ca-ad1a-b0a1d82a68a7 None admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:32.328191 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2e5816-e1ca-45ca-ad1a-b0a1d82a68a7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:32.330590 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2e5816-e1ca-45ca-ad1a-b0a1d82a68a7 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:32.331179 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bb30fb-b2fd-4b01-aefa-e9d334687e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:32.331916 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bb30fb-b2fd-4b01-aefa-e9d334687e53 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:32.332264 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bb30fb-b2fd-4b01-aefa-e9d334687e53 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:32.332611 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bb30fb-b2fd-4b01-aefa-e9d334687e53 None admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:32.333351 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bb30fb-b2fd-4b01-aefa-e9d334687e53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:32.333756 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bb30fb-b2fd-4b01-aefa-e9d334687e53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:32.335237 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bb30fb-b2fd-4b01-aefa-e9d334687e53 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:32.353827 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 230/469] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 15 07:20:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:32.353827 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bb30fb-b2fd-4b01-aefa-e9d334687e53 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:32.440534 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 240/470] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Sep 15 07:20:32 2022] POST /identity/v3/users => generated 248 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:20:32.440534 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2e5816-e1ca-45ca-ad1a-b0a1d82a68a7 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:32.475450 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bb30fb-b2fd-4b01-aefa-e9d334687e53 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:32.515574 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b46b96-a3d5-40df-ae0a-1b9b5ded0364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:32.516192 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b46b96-a3d5-40df-ae0a-1b9b5ded0364 None admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:32.516478 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b46b96-a3d5-40df-ae0a-1b9b5ded0364 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:32.516749 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b46b96-a3d5-40df-ae0a-1b9b5ded0364 None admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:32.517412 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b46b96-a3d5-40df-ae0a-1b9b5ded0364 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:32.519029 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b46b96-a3d5-40df-ae0a-1b9b5ded0364 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:32.552493 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2e5816-e1ca-45ca-ad1a-b0a1d82a68a7 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:32.589105 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe4f515-2d12-4fe3-afc4-71f7d3db3922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:32.589759 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe4f515-2d12-4fe3-afc4-71f7d3db3922 None admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:32.590044 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe4f515-2d12-4fe3-afc4-71f7d3db3922 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:32.590317 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe4f515-2d12-4fe3-afc4-71f7d3db3922 None admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:32.590979 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe4f515-2d12-4fe3-afc4-71f7d3db3922 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:32.592467 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe4f515-2d12-4fe3-afc4-71f7d3db3922 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:32.606034 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 231/471] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Sep 15 07:20:32 2022] POST /identity/v3/users => generated 248 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:20:32.606034 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b46b96-a3d5-40df-ae0a-1b9b5ded0364 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:32.610830 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:32.711542 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 241/472] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Sep 15 07:20:32 2022] POST /identity/v3/users => generated 246 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:20:32.713197 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4c8857-a16c-48ba-b4fd-46ce44072a9d None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:32.714603 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4c8857-a16c-48ba-b4fd-46ce44072a9d None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:32.714895 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4c8857-a16c-48ba-b4fd-46ce44072a9d None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:32.716304 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 242/473] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:20:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:20:32.723468 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa939057-143c-426a-b609-f99b456d1ad0 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:32.723783 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa939057-143c-426a-b609-f99b456d1ad0 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:32.724026 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa939057-143c-426a-b609-f99b456d1ad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:32.745269 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b46b96-a3d5-40df-ae0a-1b9b5ded0364 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:32.746384 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-fa939057-143c-426a-b609-f99b456d1ad0 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:20:32.747845 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-fa939057-143c-426a-b609-f99b456d1ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:20:32.784518 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:32.804848 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bdca13-0f4c-49bc-b28a-47c446089235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:32.805522 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bdca13-0f4c-49bc-b28a-47c446089235 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:32.805829 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bdca13-0f4c-49bc-b28a-47c446089235 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:32.806107 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bdca13-0f4c-49bc-b28a-47c446089235 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:32.820735 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bdca13-0f4c-49bc-b28a-47c446089235 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:32.822246 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bdca13-0f4c-49bc-b28a-47c446089235 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:32.839039 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-10bdca13-0f4c-49bc-b28a-47c446089235 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:20:32.839039 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:32.839039 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:32.839039 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:32.839039 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:32.839039 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:32.839039 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:32.839039 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:32.839039 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:32.839039 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:32.839039 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:32.839039 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:32.839039 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 07:20:32.839039 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 07:20:32.839039 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 07:20:32.839039 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 07:20:32.839998 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:32.839998 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:32.839998 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:32.839998 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:32.839998 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:32.839998 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:32.839998 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:32.839998 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:32.839998 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:32.839998 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:32.839998 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:32.839998 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:32.839998 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:32.839998 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:32.839998 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:32.839998 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:32.839998 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 07:20:32.840816 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 07:20:32.840816 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:20:32.840816 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:20:32.840816 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 232/474] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:20:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:32.848021 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bdca13-0f4c-49bc-b28a-47c446089235 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:32.872940 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 243/475] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:20:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 07:20:32.925380 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:32.979100 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bdca13-0f4c-49bc-b28a-47c446089235 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:33.017877 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a801f23e-cf55-4600-ae3c-7d420d1a8fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:33.018484 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a801f23e-cf55-4600-ae3c-7d420d1a8fd0 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:33.018765 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a801f23e-cf55-4600-ae3c-7d420d1a8fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:33.019041 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a801f23e-cf55-4600-ae3c-7d420d1a8fd0 None admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:33.019744 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a801f23e-cf55-4600-ae3c-7d420d1a8fd0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:33.020050 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a801f23e-cf55-4600-ae3c-7d420d1a8fd0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:33.021409 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a801f23e-cf55-4600-ae3c-7d420d1a8fd0 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:33.034735 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 233/476] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 15 07:20:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:20:33.040189 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa939057-143c-426a-b609-f99b456d1ad0 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:33.165776 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa939057-143c-426a-b609-f99b456d1ad0 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:33.192030 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a801f23e-cf55-4600-ae3c-7d420d1a8fd0 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:33.201443 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45f2b15-1dc2-4c46-bd60-77d9d50dc758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:33.203796 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45f2b15-1dc2-4c46-bd60-77d9d50dc758 None admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:33.204257 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45f2b15-1dc2-4c46-bd60-77d9d50dc758 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:33.210103 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45f2b15-1dc2-4c46-bd60-77d9d50dc758 None admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:33.210969 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45f2b15-1dc2-4c46-bd60-77d9d50dc758 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:33.212610 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45f2b15-1dc2-4c46-bd60-77d9d50dc758 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:33.316824 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 244/477] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Sep 15 07:20:33 2022] POST /identity/v3/users => generated 249 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:20:33.342537 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a801f23e-cf55-4600-ae3c-7d420d1a8fd0 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:33.360040 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:33.389863 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1325f7e8-98ef-45e5-ae1b-91be94607ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:33.390903 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1325f7e8-98ef-45e5-ae1b-91be94607ffb None admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:33.391360 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1325f7e8-98ef-45e5-ae1b-91be94607ffb None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:33.391782 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1325f7e8-98ef-45e5-ae1b-91be94607ffb None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:33.392642 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1325f7e8-98ef-45e5-ae1b-91be94607ffb None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:33.394197 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1325f7e8-98ef-45e5-ae1b-91be94607ffb None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:33.446105 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.sql.core [None req-1325f7e8-98ef-45e5-ae1b-91be94607ffb None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 15 07:20:33.446105 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 15 07:20:33.446105 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [parameters: {'id': '0ddffc7481604ccf927efe2969e1826a', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 15 07:20:33.446105 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81597) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 15 07:20:33.467684 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-1325f7e8-98ef-45e5-ae1b-91be94607ffb None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 15 07:20:33.467684 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:33.467684 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 15 07:20:33.467684 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 15 07:20:33.467684 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 15 07:20:33.467684 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 15 07:20:33.467684 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 15 07:20:33.467684 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application result = self._query(query) Sep 15 07:20:33.467684 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 15 07:20:33.467684 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application conn.query(q) Sep 15 07:20:33.467684 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 15 07:20:33.467684 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 15 07:20:33.467684 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 15 07:20:33.467684 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application result.read() Sep 15 07:20:33.467684 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 15 07:20:33.468809 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 15 07:20:33.468809 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 15 07:20:33.468809 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 15 07:20:33.468809 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 15 07:20:33.468809 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 15 07:20:33.468809 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 15 07:20:33.468809 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 15 07:20:33.468809 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 15 07:20:33.468809 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:20:33.468809 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 15 07:20:33.468809 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:20:33.468809 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:33.468809 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 15 07:20:33.468809 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 15 07:20:33.468809 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 15 07:20:33.468809 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return ref.to_dict() Sep 15 07:20:33.468809 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 15 07:20:33.468809 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application next(self.gen) Sep 15 07:20:33.469961 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 15 07:20:33.469961 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application yield resource Sep 15 07:20:33.469961 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 15 07:20:33.469961 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application next(self.gen) Sep 15 07:20:33.469961 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 15 07:20:33.469961 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application self.session.rollback() Sep 15 07:20:33.469961 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 15 07:20:33.469961 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application self.force_reraise() Sep 15 07:20:33.469961 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 15 07:20:33.469961 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise self.value Sep 15 07:20:33.469961 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 15 07:20:33.469961 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 15 07:20:33.469961 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 15 07:20:33.469961 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application session.commit() Sep 15 07:20:33.469961 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 15 07:20:33.469961 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 15 07:20:33.469961 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 15 07:20:33.471020 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application self._prepare_impl() Sep 15 07:20:33.471020 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 15 07:20:33.471020 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application self.session.flush() Sep 15 07:20:33.471020 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 15 07:20:33.471020 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application self._flush(objects) Sep 15 07:20:33.471020 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 15 07:20:33.471020 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 15 07:20:33.471020 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 15 07:20:33.471020 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application compat.raise_( Sep 15 07:20:33.471020 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 15 07:20:33.471020 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception Sep 15 07:20:33.471020 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 15 07:20:33.471020 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application flush_context.execute() Sep 15 07:20:33.471020 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 15 07:20:33.471020 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rec.execute(self) Sep 15 07:20:33.471020 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 15 07:20:33.471020 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 15 07:20:33.472118 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 15 07:20:33.472118 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 15 07:20:33.472118 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 15 07:20:33.472118 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 15 07:20:33.472118 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 15 07:20:33.472118 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 15 07:20:33.472118 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 15 07:20:33.472118 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 15 07:20:33.472118 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 15 07:20:33.472118 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 15 07:20:33.472118 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 15 07:20:33.472118 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 15 07:20:33.472118 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 15 07:20:33.472118 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 15 07:20:33.472118 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 15 07:20:33.472118 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception Sep 15 07:20:33.473117 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 15 07:20:33.473117 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 15 07:20:33.473117 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 15 07:20:33.473117 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 15 07:20:33.473117 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 15 07:20:33.473117 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application result = self._query(query) Sep 15 07:20:33.473117 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 15 07:20:33.473117 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application conn.query(q) Sep 15 07:20:33.473117 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 15 07:20:33.473117 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 15 07:20:33.473117 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 15 07:20:33.473117 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application result.read() Sep 15 07:20:33.473117 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 15 07:20:33.473117 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 15 07:20:33.473117 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 15 07:20:33.473117 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 15 07:20:33.473991 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 15 07:20:33.473991 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 15 07:20:33.473991 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 15 07:20:33.473991 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 15 07:20:33.473991 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 15 07:20:33.473991 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 15 07:20:33.473991 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [parameters: {'id': '0ddffc7481604ccf927efe2969e1826a', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 15 07:20:33.473991 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 15 07:20:33.473991 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:20:33.473991 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 07:20:33.473991 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:20:33.473991 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:33.473991 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:33.473991 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:33.473991 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:33.473991 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:33.475065 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:33.475065 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:33.475065 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:33.475065 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:33.475065 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:33.475065 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:33.475065 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 15 07:20:33.475065 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 15 07:20:33.475065 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:33.475065 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:33.475065 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 15 07:20:33.475065 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 15 07:20:33.475065 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 15 07:20:33.475065 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 15 07:20:33.475065 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 15 07:20:33.475065 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:20:33.477311 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 234/478] 10.222.0.42 () {60 vars in 1131 bytes} [Thu Sep 15 07:20:33 2022] POST /identity/v3/roles => generated 144 bytes in 286 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:33.484421 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45f2b15-1dc2-4c46-bd60-77d9d50dc758 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:33.511119 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:33.623806 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45f2b15-1dc2-4c46-bd60-77d9d50dc758 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:33.663527 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8182316-31b7-429f-b730-59dbcdf96fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:33.664413 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8182316-31b7-429f-b730-59dbcdf96fb7 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:33.664850 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8182316-31b7-429f-b730-59dbcdf96fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:33.665270 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8182316-31b7-429f-b730-59dbcdf96fb7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:33.677475 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8182316-31b7-429f-b730-59dbcdf96fb7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:20:33.683252 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-d8182316-31b7-429f-b730-59dbcdf96fb7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 07:20:33.683252 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:33.683252 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:33.683252 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:33.683252 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:33.683252 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:33.683252 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:33.683252 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:33.683252 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:33.683252 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:33.683252 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:33.683252 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:33.683252 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:20:33.683252 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:20:33.683252 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:20:33.683252 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:20:33.684316 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:20:33.684316 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:20:33.684316 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:20:33.684316 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:20:33.684316 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:33.684316 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:33.684316 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:33.684316 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:33.684316 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:33.684316 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:33.684316 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:33.684316 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:33.684316 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:33.684316 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:33.684316 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:33.684316 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:33.685326 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:33.685326 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:33.685326 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:33.685326 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:33.685326 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:20:33.685326 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:20:33.685326 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:20:33.685326 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:20:33.685326 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:20:33.685326 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:20:33.685326 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 07:20:33.685326 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:20:33.687614 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 245/479] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Sep 15 07:20:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 07:20:33.693461 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1325f7e8-98ef-45e5-ae1b-91be94607ffb None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:33.814816 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1325f7e8-98ef-45e5-ae1b-91be94607ffb None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:33.851843 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9c5a2a-100c-4c31-96d5-636ba64da1f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:33.852691 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9c5a2a-100c-4c31-96d5-636ba64da1f3 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:33.853122 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9c5a2a-100c-4c31-96d5-636ba64da1f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:33.853570 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9c5a2a-100c-4c31-96d5-636ba64da1f3 None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:33.854425 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9c5a2a-100c-4c31-96d5-636ba64da1f3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:33.854886 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9c5a2a-100c-4c31-96d5-636ba64da1f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:33.856416 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9c5a2a-100c-4c31-96d5-636ba64da1f3 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:33.870086 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 235/480] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 15 07:20:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:35.164782 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84e3769-e9d9-49b3-b7e9-0a35e3fa989d None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:35.165292 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84e3769-e9d9-49b3-b7e9-0a35e3fa989d None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:35.165367 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84e3769-e9d9-49b3-b7e9-0a35e3fa989d None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:35.166964 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 246/481] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:20:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:20:35.175453 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161fdb0b-b13e-4f5d-8f53-2f9c7543846a None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:35.179334 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161fdb0b-b13e-4f5d-8f53-2f9c7543846a None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:35.179334 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161fdb0b-b13e-4f5d-8f53-2f9c7543846a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:35.198965 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-161fdb0b-b13e-4f5d-8f53-2f9c7543846a None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:20:35.200107 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-161fdb0b-b13e-4f5d-8f53-2f9c7543846a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:20:35.254757 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d539c1a1-0457-484c-9f2b-69d0284d0de3 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:35.255040 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d539c1a1-0457-484c-9f2b-69d0284d0de3 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:35.255326 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d539c1a1-0457-484c-9f2b-69d0284d0de3 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:35.256652 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 247/482] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:20:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:20:35.266035 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb6e2a0-aed4-48b8-8d8f-f26c0dfca5e9 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:35.266316 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb6e2a0-aed4-48b8-8d8f-f26c0dfca5e9 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:35.266590 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb6e2a0-aed4-48b8-8d8f-f26c0dfca5e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:35.289907 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:35.292228 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-7fb6e2a0-aed4-48b8-8d8f-f26c0dfca5e9 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:20:35.293490 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-7fb6e2a0-aed4-48b8-8d8f-f26c0dfca5e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:20:35.351462 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 236/483] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:20:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 07:20:35.397485 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 248/484] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:20:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 07:20:35.466333 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cd08ce-ccfb-40f9-8d4c-7838d086ae37 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:35.466586 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cd08ce-ccfb-40f9-8d4c-7838d086ae37 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:35.466814 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cd08ce-ccfb-40f9-8d4c-7838d086ae37 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:35.469081 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 237/485] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:20:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:20:35.474404 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99a5ab-54d6-4bf7-9171-866cfd84e38b None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:35.474818 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99a5ab-54d6-4bf7-9171-866cfd84e38b None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:35.475187 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99a5ab-54d6-4bf7-9171-866cfd84e38b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:35.500189 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-8c99a5ab-54d6-4bf7-9171-866cfd84e38b None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:20:35.501361 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-8c99a5ab-54d6-4bf7-9171-866cfd84e38b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:20:35.503392 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:35.605105 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 249/486] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:20:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 07:20:35.699173 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cd08ce-ccfb-40f9-8d4c-7838d086ae37 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:35.767759 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c99a5ab-54d6-4bf7-9171-866cfd84e38b None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:35.816819 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cd08ce-ccfb-40f9-8d4c-7838d086ae37 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:35.866279 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62062600-b4ea-4bdf-9b07-619c4f92d8a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:35.866926 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62062600-b4ea-4bdf-9b07-619c4f92d8a2 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:35.867215 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62062600-b4ea-4bdf-9b07-619c4f92d8a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:35.867496 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62062600-b4ea-4bdf-9b07-619c4f92d8a2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:35.901126 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c99a5ab-54d6-4bf7-9171-866cfd84e38b None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:35.910186 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62062600-b4ea-4bdf-9b07-619c4f92d8a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:20:35.915127 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-62062600-b4ea-4bdf-9b07-619c4f92d8a2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 07:20:35.915127 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:35.915127 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:35.915127 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:35.915127 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:35.915127 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:35.915127 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:35.915127 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:35.915127 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:35.915127 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:35.915127 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:35.915127 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:35.915127 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:20:35.915127 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:20:35.915127 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:20:35.915127 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:20:35.915999 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:20:35.915999 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:20:35.915999 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:20:35.915999 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:20:35.915999 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:35.915999 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:35.915999 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:35.915999 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:35.915999 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:35.915999 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:35.915999 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:35.915999 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:35.915999 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:35.915999 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:35.915999 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:35.915999 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:35.916788 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:35.916788 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:35.916788 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:35.916788 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:35.916788 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:20:35.916788 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:20:35.916788 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:20:35.916788 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:20:35.916788 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:20:35.916788 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:20:35.916788 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 07:20:35.916788 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:20:35.916788 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 238/487] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Sep 15 07:20:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:35.922646 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62062600-b4ea-4bdf-9b07-619c4f92d8a2 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:35.954208 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae97a0a0-8f13-466b-b6bb-9110cddfcbcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:35.955650 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae97a0a0-8f13-466b-b6bb-9110cddfcbcf None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:35.956073 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae97a0a0-8f13-466b-b6bb-9110cddfcbcf None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:35.956528 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae97a0a0-8f13-466b-b6bb-9110cddfcbcf None admin] PATH_INFO: `/v3/roles/service` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:35.969042 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae97a0a0-8f13-466b-b6bb-9110cddfcbcf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:20:35.977842 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:35.980143 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-ae97a0a0-8f13-466b-b6bb-9110cddfcbcf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 07:20:35.980143 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:35.980143 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:35.980143 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:35.980143 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:35.980143 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:35.980143 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:35.980143 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:35.980143 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:35.980143 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:35.980143 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:35.980143 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:35.980143 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:20:35.980143 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:20:35.980143 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:20:35.980143 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:20:35.981286 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:20:35.981286 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:20:35.981286 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:20:35.981286 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:20:35.981286 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:35.981286 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:35.981286 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:35.981286 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:35.981286 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:35.981286 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:35.981286 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:35.981286 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:35.981286 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:35.981286 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:35.981286 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:35.981286 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:35.982302 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:35.982302 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:35.982302 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:35.982302 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:35.982302 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:20:35.982302 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:20:35.982302 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:20:35.982302 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:20:35.982302 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:20:35.982302 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:20:35.982302 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 07:20:35.982302 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:20:35.986153 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 250/488] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Sep 15 07:20:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:35.992481 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae97a0a0-8f13-466b-b6bb-9110cddfcbcf None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:36.003021 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:36.081446 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62062600-b4ea-4bdf-9b07-619c4f92d8a2 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:36.117391 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae97a0a0-8f13-466b-b6bb-9110cddfcbcf None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:36.133222 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73177a6-c16a-4982-b972-391c90c5c491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:36.133873 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73177a6-c16a-4982-b972-391c90c5c491 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:36.134153 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73177a6-c16a-4982-b972-391c90c5c491 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:36.134469 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73177a6-c16a-4982-b972-391c90c5c491 None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:36.135095 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73177a6-c16a-4982-b972-391c90c5c491 None admin] RBAC: Adding query filter params (name=service) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:36.135403 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73177a6-c16a-4982-b972-391c90c5c491 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:36.136900 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73177a6-c16a-4982-b972-391c90c5c491 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:36.149808 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 239/489] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Sep 15 07:20:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:36.150945 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73177a6-c16a-4982-b972-391c90c5c491 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:36.160699 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f039ed1-e917-4f88-b0a0-5f3ca1c7104a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:36.161615 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f039ed1-e917-4f88-b0a0-5f3ca1c7104a None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:36.162567 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f039ed1-e917-4f88-b0a0-5f3ca1c7104a None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:36.165362 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f039ed1-e917-4f88-b0a0-5f3ca1c7104a None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:36.165362 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f039ed1-e917-4f88-b0a0-5f3ca1c7104a None admin] RBAC: Adding query filter params (name=service) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:36.165362 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f039ed1-e917-4f88-b0a0-5f3ca1c7104a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:36.166849 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f039ed1-e917-4f88-b0a0-5f3ca1c7104a None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:36.177129 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 251/490] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Sep 15 07:20:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:20:36.178337 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f039ed1-e917-4f88-b0a0-5f3ca1c7104a None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:36.243532 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:36.259190 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:36.268145 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73177a6-c16a-4982-b972-391c90c5c491 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:36.290010 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f039ed1-e917-4f88-b0a0-5f3ca1c7104a None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:36.306760 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75278dee-f5fc-4535-8452-5e0ea5c88eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:36.307484 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75278dee-f5fc-4535-8452-5e0ea5c88eb7 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:36.307837 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75278dee-f5fc-4535-8452-5e0ea5c88eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:36.308461 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75278dee-f5fc-4535-8452-5e0ea5c88eb7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:36.319945 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75278dee-f5fc-4535-8452-5e0ea5c88eb7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:20:36.325178 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-75278dee-f5fc-4535-8452-5e0ea5c88eb7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 07:20:36.325178 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:36.325178 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:36.325178 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:36.325178 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:36.325178 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:36.325178 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:36.325178 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:36.325178 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:36.325178 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:36.325178 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:36.325178 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:36.325178 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:20:36.325178 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:20:36.325178 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:20:36.325178 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:20:36.326120 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:20:36.326120 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:20:36.326120 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:20:36.326120 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:20:36.326120 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:36.326120 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:36.326120 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:36.326120 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:36.326120 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:36.326120 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:36.326120 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:36.326120 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:36.326120 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:36.326120 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:36.326120 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:36.326120 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:36.327434 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:36.327434 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:36.327434 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:36.327434 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:36.327434 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:20:36.327434 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:20:36.327434 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:20:36.327434 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:20:36.327434 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:20:36.327434 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:20:36.327434 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 07:20:36.327434 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:20:36.327434 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 240/491] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Sep 15 07:20:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:36.332305 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cfb728-9536-451b-8329-21c22ded6c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:36.332991 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75278dee-f5fc-4535-8452-5e0ea5c88eb7 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:36.333826 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cfb728-9536-451b-8329-21c22ded6c21 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:36.334300 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cfb728-9536-451b-8329-21c22ded6c21 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:36.334795 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cfb728-9536-451b-8329-21c22ded6c21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:36.348979 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cfb728-9536-451b-8329-21c22ded6c21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:36.351011 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cfb728-9536-451b-8329-21c22ded6c21 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:36.369048 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-55cfb728-9536-451b-8329-21c22ded6c21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:20:36.369048 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:36.369048 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:36.369048 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:36.369048 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:36.369048 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:36.369048 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:36.369048 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:36.369048 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:36.369048 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:36.369048 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:36.369048 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:36.369048 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 07:20:36.369048 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 07:20:36.369048 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 07:20:36.369048 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 07:20:36.370054 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:36.370054 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:36.370054 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:36.370054 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:36.370054 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:36.370054 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:36.370054 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:36.370054 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:36.370054 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:36.370054 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:36.370054 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:36.370054 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:36.370054 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:36.370054 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:36.370054 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:36.370054 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:36.370054 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 07:20:36.370877 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 07:20:36.370877 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:20:36.370877 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:20:36.372774 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 252/492] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:20:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:36.373782 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cfb728-9536-451b-8329-21c22ded6c21 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:36.455010 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75278dee-f5fc-4535-8452-5e0ea5c88eb7 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:36.486184 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cfb728-9536-451b-8329-21c22ded6c21 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:36.491340 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6296f827-a20a-4b8a-9d8d-2edff31d0026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:36.491974 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6296f827-a20a-4b8a-9d8d-2edff31d0026 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:36.492255 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6296f827-a20a-4b8a-9d8d-2edff31d0026 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:36.492529 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6296f827-a20a-4b8a-9d8d-2edff31d0026 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:36.502034 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:36.505120 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6296f827-a20a-4b8a-9d8d-2edff31d0026 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:36.506666 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6296f827-a20a-4b8a-9d8d-2edff31d0026 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:36.524322 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-6296f827-a20a-4b8a-9d8d-2edff31d0026 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:20:36.524322 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:36.524322 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:36.524322 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:36.524322 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:36.524322 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:36.524322 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:36.524322 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:36.524322 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:36.524322 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:36.524322 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:36.524322 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:36.524322 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 07:20:36.524322 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 07:20:36.524322 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 07:20:36.524322 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 07:20:36.525722 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:36.525722 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:36.525722 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:36.525722 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:36.525722 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:36.525722 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:36.525722 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:36.525722 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:36.525722 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:36.525722 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:36.525722 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:36.525722 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:36.525722 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:36.525722 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:36.525722 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:36.525722 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:36.525722 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 07:20:36.526644 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 07:20:36.526644 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:20:36.526644 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:20:36.526808 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc6a61c-d1e7-404e-ab68-05abcbaf05dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:36.526808 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc6a61c-d1e7-404e-ab68-05abcbaf05dc None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:36.526808 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc6a61c-d1e7-404e-ab68-05abcbaf05dc None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:36.526808 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc6a61c-d1e7-404e-ab68-05abcbaf05dc None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:36.527428 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc6a61c-d1e7-404e-ab68-05abcbaf05dc None admin] RBAC: Adding query filter params (name=service) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:36.527533 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc6a61c-d1e7-404e-ab68-05abcbaf05dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:36.528865 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc6a61c-d1e7-404e-ab68-05abcbaf05dc None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:36.531514 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 241/493] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:20:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:36.531514 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6296f827-a20a-4b8a-9d8d-2edff31d0026 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:36.535500 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:36.544941 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 253/494] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Sep 15 07:20:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:36.544941 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4ad85a-ae8d-4a75-9c9e-72b2e6aac51b None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:36.544941 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4ad85a-ae8d-4a75-9c9e-72b2e6aac51b None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:36.544941 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4ad85a-ae8d-4a75-9c9e-72b2e6aac51b None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:36.544941 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 254/495] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:20:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:20:36.544941 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4ad85a-ae8d-4a75-9c9e-72b2e6aac51b None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:36.646569 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6296f827-a20a-4b8a-9d8d-2edff31d0026 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:36.650413 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4ad85a-ae8d-4a75-9c9e-72b2e6aac51b None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:36.680404 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c69b98-eaa7-4920-b92e-4eff68434388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:36.681007 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c69b98-eaa7-4920-b92e-4eff68434388 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:36.681289 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c69b98-eaa7-4920-b92e-4eff68434388 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:36.681563 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c69b98-eaa7-4920-b92e-4eff68434388 None admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:36.682308 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c69b98-eaa7-4920-b92e-4eff68434388 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:36.682599 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c69b98-eaa7-4920-b92e-4eff68434388 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:36.684225 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c69b98-eaa7-4920-b92e-4eff68434388 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:36.684831 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12afb76-c1b1-4137-a4b8-b32293db8b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:36.685435 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12afb76-c1b1-4137-a4b8-b32293db8b25 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:36.685749 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12afb76-c1b1-4137-a4b8-b32293db8b25 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:36.686026 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12afb76-c1b1-4137-a4b8-b32293db8b25 None admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:36.687461 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12afb76-c1b1-4137-a4b8-b32293db8b25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:36.687886 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12afb76-c1b1-4137-a4b8-b32293db8b25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:36.689250 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12afb76-c1b1-4137-a4b8-b32293db8b25 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:36.696909 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 242/496] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 15 07:20:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:36.697780 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c69b98-eaa7-4920-b92e-4eff68434388 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:36.704003 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 255/497] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 15 07:20:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:36.705246 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d59936-f49d-4047-8016-ffcb0de23786 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:36.705531 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d59936-f49d-4047-8016-ffcb0de23786 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:36.705838 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d59936-f49d-4047-8016-ffcb0de23786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:36.723911 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-d0d59936-f49d-4047-8016-ffcb0de23786 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:20:36.725089 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-d0d59936-f49d-4047-8016-ffcb0de23786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:20:36.752541 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:36.763753 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:36.792333 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 256/498] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:20:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 07:20:36.793457 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1a77e6-1f07-4a39-802e-d7b7004a95af None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:36.793833 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1a77e6-1f07-4a39-802e-d7b7004a95af None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:36.794171 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1a77e6-1f07-4a39-802e-d7b7004a95af None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:36.795454 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 257/499] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:20:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:20:36.796281 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1a77e6-1f07-4a39-802e-d7b7004a95af None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:36.812540 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c69b98-eaa7-4920-b92e-4eff68434388 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:36.849048 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaff90d-027f-4795-9234-ea6151f50d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:36.849712 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaff90d-027f-4795-9234-ea6151f50d8a None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:36.850003 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaff90d-027f-4795-9234-ea6151f50d8a None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:36.850281 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaff90d-027f-4795-9234-ea6151f50d8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:36.861912 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaff90d-027f-4795-9234-ea6151f50d8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:36.863448 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaff90d-027f-4795-9234-ea6151f50d8a None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:36.878638 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-3aaff90d-027f-4795-9234-ea6151f50d8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:20:36.878638 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:36.878638 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:36.878638 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:36.878638 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:36.878638 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:36.878638 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:36.878638 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:36.878638 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:36.878638 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:36.878638 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:36.878638 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:36.878638 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 07:20:36.878638 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 07:20:36.878638 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 07:20:36.878638 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 07:20:36.879622 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:36.879622 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:36.879622 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:36.879622 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:36.879622 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:36.879622 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:36.879622 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:36.879622 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:36.879622 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:36.879622 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:36.879622 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:36.879622 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:36.879622 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:36.879622 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:36.879622 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:36.879622 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:36.879622 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 07:20:36.880779 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 07:20:36.880779 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:20:36.880779 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:20:36.880779 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 243/500] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:20:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:36.880779 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaff90d-027f-4795-9234-ea6151f50d8a None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:36.911179 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1a77e6-1f07-4a39-802e-d7b7004a95af None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:36.951404 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cbfec3-1eae-47ff-9651-e925a7faeb80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:36.951771 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cbfec3-1eae-47ff-9651-e925a7faeb80 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:36.952687 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cbfec3-1eae-47ff-9651-e925a7faeb80 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:36.953163 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cbfec3-1eae-47ff-9651-e925a7faeb80 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:36.960431 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cbfec3-1eae-47ff-9651-e925a7faeb80 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:36.961907 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cbfec3-1eae-47ff-9651-e925a7faeb80 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:36.975715 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-d8cbfec3-1eae-47ff-9651-e925a7faeb80 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 15 07:20:36.975715 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:36.975715 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:36.975715 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:36.975715 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:36.975715 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:36.975715 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:36.975715 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:36.975715 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:36.975715 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:36.975715 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:36.975715 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:36.975715 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 07:20:36.975715 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 07:20:36.975715 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 07:20:36.975715 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 07:20:36.976612 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:36.976612 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:36.976612 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 07:20:36.976612 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 07:20:36.976612 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 07:20:36.976612 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 07:20:36.976612 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:36.976612 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:36.976612 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:36.976612 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:36.976612 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:36.976612 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:36.976612 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:36.976612 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:36.976612 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:36.976612 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:36.976612 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:36.976612 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:36.977491 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:36.977491 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:36.977491 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 07:20:36.977491 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 07:20:36.977491 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 07:20:36.977491 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 07:20:36.977491 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 07:20:36.977491 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 07:20:36.977491 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 15 07:20:36.977491 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:20:36.981493 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 258/501] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 15 07:20:36 2022] GET /identity/v3/users/glance => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:36.982722 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa52f87-dd6d-460e-bda8-d19c37e429e0 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:36.983007 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa52f87-dd6d-460e-bda8-d19c37e429e0 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:36.983959 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa52f87-dd6d-460e-bda8-d19c37e429e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:36.995544 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaff90d-027f-4795-9234-ea6151f50d8a None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:37.005940 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-eaa52f87-dd6d-460e-bda8-d19c37e429e0 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:20:37.007505 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-eaa52f87-dd6d-460e-bda8-d19c37e429e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:20:37.014602 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:37.025967 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:37.034110 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830df83b-4696-457d-9646-66b56878c875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:37.035164 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830df83b-4696-457d-9646-66b56878c875 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:37.035967 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830df83b-4696-457d-9646-66b56878c875 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:37.036699 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830df83b-4696-457d-9646-66b56878c875 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:37.045546 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830df83b-4696-457d-9646-66b56878c875 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:37.047508 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830df83b-4696-457d-9646-66b56878c875 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:37.063963 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-830df83b-4696-457d-9646-66b56878c875 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 15 07:20:37.063963 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:37.063963 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:37.063963 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:37.063963 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:37.063963 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:37.063963 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:37.063963 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:37.063963 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:37.063963 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:37.063963 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:37.063963 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:37.063963 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 07:20:37.063963 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 07:20:37.063963 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 07:20:37.063963 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 07:20:37.065051 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:37.065051 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:37.065051 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 07:20:37.065051 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 07:20:37.065051 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 07:20:37.065051 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 07:20:37.065051 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:37.065051 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:37.065051 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:37.065051 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:37.065051 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:37.065051 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:37.065051 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:37.065051 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:37.065051 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:37.065051 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:37.065051 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:37.065051 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:37.066083 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:37.066083 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:37.066083 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 07:20:37.066083 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 07:20:37.066083 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 07:20:37.066083 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 07:20:37.066083 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 07:20:37.066083 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 07:20:37.066083 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 15 07:20:37.066083 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:20:37.066083 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 244/502] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 15 07:20:36 2022] GET /identity/v3/users/cinder => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:37.066774 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830df83b-4696-457d-9646-66b56878c875 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:37.085613 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 259/503] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:20:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 07:20:37.085613 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa52f87-dd6d-460e-bda8-d19c37e429e0 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:37.172908 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830df83b-4696-457d-9646-66b56878c875 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:37.184409 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa52f87-dd6d-460e-bda8-d19c37e429e0 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:37.208803 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dd5149-bb62-4fdf-85e3-b3fa95ef7951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:37.209419 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dd5149-bb62-4fdf-85e3-b3fa95ef7951 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:37.210053 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dd5149-bb62-4fdf-85e3-b3fa95ef7951 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:37.210481 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dd5149-bb62-4fdf-85e3-b3fa95ef7951 None admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:37.211108 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dd5149-bb62-4fdf-85e3-b3fa95ef7951 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:37.211420 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dd5149-bb62-4fdf-85e3-b3fa95ef7951 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:37.212796 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dd5149-bb62-4fdf-85e3-b3fa95ef7951 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:37.220573 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52ac0de-7bb7-4586-aa17-53b7b2696294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:37.221181 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ac0de-7bb7-4586-aa17-53b7b2696294 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:37.221458 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ac0de-7bb7-4586-aa17-53b7b2696294 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:37.221763 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ac0de-7bb7-4586-aa17-53b7b2696294 None admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:37.222482 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52ac0de-7bb7-4586-aa17-53b7b2696294 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:37.222788 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52ac0de-7bb7-4586-aa17-53b7b2696294 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:37.224196 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52ac0de-7bb7-4586-aa17-53b7b2696294 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:37.229955 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 245/504] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 15 07:20:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:37.229955 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dd5149-bb62-4fdf-85e3-b3fa95ef7951 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:37.270306 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 260/505] 10.222.0.42 () {56 vars in 1137 bytes} [Thu Sep 15 07:20:37 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:37.270306 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52ac0de-7bb7-4586-aa17-53b7b2696294 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:37.271966 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:37.281778 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:37.342454 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dd5149-bb62-4fdf-85e3-b3fa95ef7951 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:37.371332 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52ac0de-7bb7-4586-aa17-53b7b2696294 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:37.378023 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3487509d-2fa0-4d5f-a2b0-f7d3cb564ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:37.378656 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3487509d-2fa0-4d5f-a2b0-f7d3cb564ea5 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:37.379332 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3487509d-2fa0-4d5f-a2b0-f7d3cb564ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:37.379629 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3487509d-2fa0-4d5f-a2b0-f7d3cb564ea5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:37.390831 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3487509d-2fa0-4d5f-a2b0-f7d3cb564ea5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:20:37.395649 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-3487509d-2fa0-4d5f-a2b0-f7d3cb564ea5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 07:20:37.395649 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:37.395649 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:37.395649 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:37.395649 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:37.395649 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:37.395649 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:37.395649 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:37.395649 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:37.395649 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:37.395649 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:37.395649 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:37.395649 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:20:37.395649 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:20:37.395649 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:20:37.395649 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:20:37.396497 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:20:37.396497 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:20:37.396497 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:20:37.396497 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:20:37.396497 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:37.396497 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:37.396497 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:37.396497 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:37.396497 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:37.396497 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:37.396497 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:37.396497 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:37.396497 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:37.396497 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:37.396497 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:37.396497 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:37.397279 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:37.397279 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:37.397279 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:37.397279 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:37.397279 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:20:37.397279 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:20:37.397279 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:20:37.397279 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:20:37.397279 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:20:37.397279 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:20:37.397279 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 07:20:37.397279 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:20:37.397279 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 246/506] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Sep 15 07:20:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:37.397977 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3487509d-2fa0-4d5f-a2b0-f7d3cb564ea5 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:37.406937 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157c230f-0b9a-477e-9351-c9af2a3b3954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:37.407548 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157c230f-0b9a-477e-9351-c9af2a3b3954 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:37.407833 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157c230f-0b9a-477e-9351-c9af2a3b3954 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:37.408111 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157c230f-0b9a-477e-9351-c9af2a3b3954 None admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:37.408843 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157c230f-0b9a-477e-9351-c9af2a3b3954 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:37.409162 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157c230f-0b9a-477e-9351-c9af2a3b3954 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:37.410661 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157c230f-0b9a-477e-9351-c9af2a3b3954 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:37.430244 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 261/507] 10.222.0.42 () {56 vars in 1137 bytes} [Thu Sep 15 07:20:37 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:37.430890 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157c230f-0b9a-477e-9351-c9af2a3b3954 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:37.492575 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:37.498186 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:37.498301 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3487509d-2fa0-4d5f-a2b0-f7d3cb564ea5 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:37.528474 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157c230f-0b9a-477e-9351-c9af2a3b3954 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:37.531947 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e752f3f-e0c9-42b8-86c8-e504c7f99b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:37.532573 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e752f3f-e0c9-42b8-86c8-e504c7f99b33 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:37.532849 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e752f3f-e0c9-42b8-86c8-e504c7f99b33 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:37.533117 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e752f3f-e0c9-42b8-86c8-e504c7f99b33 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:37.540034 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e752f3f-e0c9-42b8-86c8-e504c7f99b33 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:37.541450 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e752f3f-e0c9-42b8-86c8-e504c7f99b33 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:37.553532 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-4e752f3f-e0c9-42b8-86c8-e504c7f99b33 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 15 07:20:37.553532 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:37.553532 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:37.553532 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:37.553532 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:37.553532 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:37.553532 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:37.553532 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:37.553532 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:37.553532 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:37.553532 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:37.553532 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:37.553532 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 07:20:37.553532 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 07:20:37.553532 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 07:20:37.553532 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 07:20:37.554673 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:37.554673 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:37.554673 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 07:20:37.554673 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 07:20:37.554673 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 07:20:37.554673 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 07:20:37.554673 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:37.554673 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:37.554673 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:37.554673 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:37.554673 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:37.554673 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:37.554673 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:37.554673 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:37.554673 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:37.554673 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:37.554673 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:37.554673 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:37.555718 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:37.555718 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:37.555718 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 07:20:37.555718 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 07:20:37.555718 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 07:20:37.555718 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 07:20:37.555718 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 07:20:37.555718 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 07:20:37.555718 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 15 07:20:37.555718 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:20:37.555718 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 247/508] 10.222.0.42 () {56 vars in 1093 bytes} [Thu Sep 15 07:20:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:37.556386 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e752f3f-e0c9-42b8-86c8-e504c7f99b33 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:37.566937 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4546db-7510-4ea8-be36-5ae38f811d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:37.567542 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4546db-7510-4ea8-be36-5ae38f811d6a None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:37.567819 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4546db-7510-4ea8-be36-5ae38f811d6a None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:37.568087 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4546db-7510-4ea8-be36-5ae38f811d6a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:37.635856 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4546db-7510-4ea8-be36-5ae38f811d6a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:37.637292 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4546db-7510-4ea8-be36-5ae38f811d6a None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:37.660327 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e752f3f-e0c9-42b8-86c8-e504c7f99b33 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:37.693532 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5193ff4a-be26-4343-9ade-91a1d8baaa4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:37.694106 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5193ff4a-be26-4343-9ade-91a1d8baaa4f None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:37.694389 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5193ff4a-be26-4343-9ade-91a1d8baaa4f None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:37.694665 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5193ff4a-be26-4343-9ade-91a1d8baaa4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:37.706637 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:37.707801 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5193ff4a-be26-4343-9ade-91a1d8baaa4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:37.709249 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5193ff4a-be26-4343-9ade-91a1d8baaa4f None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:37.724288 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-5193ff4a-be26-4343-9ade-91a1d8baaa4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:20:37.724288 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:37.724288 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:37.724288 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:37.724288 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:37.724288 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:37.724288 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:37.724288 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:37.724288 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:37.724288 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:37.724288 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:37.724288 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:37.724288 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 07:20:37.724288 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 07:20:37.724288 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 07:20:37.724288 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 07:20:37.725662 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:37.725662 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:37.725662 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:37.725662 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:37.725662 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:37.725662 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:37.725662 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:37.725662 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:37.725662 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:37.725662 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:37.725662 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:37.725662 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:37.725662 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:37.725662 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:37.725662 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:37.725662 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:37.725662 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 07:20:37.726758 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 07:20:37.726758 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:20:37.726758 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:20:37.729722 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 248/509] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:20:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:37.730864 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5193ff4a-be26-4343-9ade-91a1d8baaa4f None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:37.740760 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 262/510] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 15 07:20:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 07:20:37.741622 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4546db-7510-4ea8-be36-5ae38f811d6a None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:37.746446 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:37.831012 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5193ff4a-be26-4343-9ade-91a1d8baaa4f None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:37.846499 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4546db-7510-4ea8-be36-5ae38f811d6a None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:37.864897 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a63091-b453-42a9-b318-e2c22f251d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:37.865505 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a63091-b453-42a9-b318-e2c22f251d8a None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:37.865836 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a63091-b453-42a9-b318-e2c22f251d8a None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:37.866113 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a63091-b453-42a9-b318-e2c22f251d8a None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:37.866784 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a63091-b453-42a9-b318-e2c22f251d8a None admin] RBAC: Adding query filter params (name=service) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:37.867090 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a63091-b453-42a9-b318-e2c22f251d8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:37.868468 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a63091-b453-42a9-b318-e2c22f251d8a None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:37.877184 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 249/511] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Sep 15 07:20:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:37.878035 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a63091-b453-42a9-b318-e2c22f251d8a None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:37.881189 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f16541-81fc-44c5-aa2f-c320ca36daf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:37.881926 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f16541-81fc-44c5-aa2f-c320ca36daf7 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:37.882267 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f16541-81fc-44c5-aa2f-c320ca36daf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:37.882549 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f16541-81fc-44c5-aa2f-c320ca36daf7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:37.924810 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:37.948788 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:37.953720 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f16541-81fc-44c5-aa2f-c320ca36daf7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:37.955916 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f16541-81fc-44c5-aa2f-c320ca36daf7 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:37.981797 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a63091-b453-42a9-b318-e2c22f251d8a None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:38.015704 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01da0512-5ac1-4473-b62b-cdcbfe59b9b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:38.016322 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01da0512-5ac1-4473-b62b-cdcbfe59b9b3 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:38.016586 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01da0512-5ac1-4473-b62b-cdcbfe59b9b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:38.016858 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01da0512-5ac1-4473-b62b-cdcbfe59b9b3 None admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:38.017593 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01da0512-5ac1-4473-b62b-cdcbfe59b9b3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:38.017906 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01da0512-5ac1-4473-b62b-cdcbfe59b9b3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:38.019318 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01da0512-5ac1-4473-b62b-cdcbfe59b9b3 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:38.054564 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 263/512] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 15 07:20:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 07:20:38.055406 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f16541-81fc-44c5-aa2f-c320ca36daf7 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:38.061516 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 250/513] 10.222.0.42 () {56 vars in 1133 bytes} [Thu Sep 15 07:20:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:38.061516 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01da0512-5ac1-4473-b62b-cdcbfe59b9b3 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:38.124751 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:38.157111 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f16541-81fc-44c5-aa2f-c320ca36daf7 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:38.157829 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01da0512-5ac1-4473-b62b-cdcbfe59b9b3 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:38.185112 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:38.193750 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76d5a33-6499-4a31-bbe8-e18cea587871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:38.194194 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9336fb-e965-40f9-ba3f-cab3849d01bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:38.194644 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76d5a33-6499-4a31-bbe8-e18cea587871 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:38.194743 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76d5a33-6499-4a31-bbe8-e18cea587871 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:38.194940 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9336fb-e965-40f9-ba3f-cab3849d01bc None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:38.195036 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76d5a33-6499-4a31-bbe8-e18cea587871 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:38.195282 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9336fb-e965-40f9-ba3f-cab3849d01bc None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:38.195564 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9336fb-e965-40f9-ba3f-cab3849d01bc None admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:38.196207 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9336fb-e965-40f9-ba3f-cab3849d01bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:38.196513 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9336fb-e965-40f9-ba3f-cab3849d01bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:38.197935 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9336fb-e965-40f9-ba3f-cab3849d01bc None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:38.205226 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76d5a33-6499-4a31-bbe8-e18cea587871 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:38.206651 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76d5a33-6499-4a31-bbe8-e18cea587871 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:38.210017 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 264/514] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 15 07:20:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:38.210902 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9336fb-e965-40f9-ba3f-cab3849d01bc None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:38.222106 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-a76d5a33-6499-4a31-bbe8-e18cea587871 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:20:38.222106 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:38.222106 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:38.222106 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:38.222106 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:38.222106 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:38.222106 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:38.222106 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:38.222106 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:38.222106 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:38.222106 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:38.222106 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:38.222106 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 07:20:38.222106 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 07:20:38.222106 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 07:20:38.222106 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 07:20:38.228189 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:38.228189 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:38.228189 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:38.228189 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:38.228189 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:38.228189 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:38.228189 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:38.228189 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:38.228189 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:38.228189 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:38.228189 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:38.228189 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:38.228189 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:38.228189 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:38.228189 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:38.228189 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:38.228189 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 07:20:38.229019 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 07:20:38.229019 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:20:38.229019 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:20:38.229019 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 251/515] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:20:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:38.229019 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76d5a33-6499-4a31-bbe8-e18cea587871 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:38.317160 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9336fb-e965-40f9-ba3f-cab3849d01bc None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:38.325915 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76d5a33-6499-4a31-bbe8-e18cea587871 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:38.345883 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:38.351139 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e89ca77-6cda-41ec-a773-008361b4c058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:38.351772 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e89ca77-6cda-41ec-a773-008361b4c058 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:38.352055 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e89ca77-6cda-41ec-a773-008361b4c058 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:38.352334 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e89ca77-6cda-41ec-a773-008361b4c058 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:38.359914 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86129780-a5a5-4101-8697-6b8b15848fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:38.361401 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86129780-a5a5-4101-8697-6b8b15848fcf None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:38.361401 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86129780-a5a5-4101-8697-6b8b15848fcf None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:38.361401 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86129780-a5a5-4101-8697-6b8b15848fcf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:38.363570 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e89ca77-6cda-41ec-a773-008361b4c058 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:38.365548 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e89ca77-6cda-41ec-a773-008361b4c058 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:38.371864 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86129780-a5a5-4101-8697-6b8b15848fcf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:38.373325 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86129780-a5a5-4101-8697-6b8b15848fcf None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:38.380389 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-5e89ca77-6cda-41ec-a773-008361b4c058 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:20:38.380389 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:38.380389 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:38.380389 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:38.380389 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:38.380389 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:38.380389 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:38.380389 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:38.380389 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:38.380389 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:38.380389 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:38.380389 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:38.380389 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 07:20:38.380389 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 07:20:38.380389 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 07:20:38.380389 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 07:20:38.381486 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:38.381486 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:38.381486 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:38.381486 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:38.381486 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:38.381486 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:38.381486 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:38.381486 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:38.381486 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:38.381486 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:38.381486 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:38.381486 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:38.381486 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:38.381486 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:38.381486 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:38.381486 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:38.381486 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 07:20:38.382479 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 07:20:38.382479 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:20:38.382479 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:20:38.386025 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 265/516] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:20:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:38.386852 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e89ca77-6cda-41ec-a773-008361b4c058 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:38.387950 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-86129780-a5a5-4101-8697-6b8b15848fcf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:20:38.387950 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:38.387950 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:38.387950 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:38.387950 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:38.387950 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:38.387950 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:38.387950 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:38.387950 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:38.387950 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:38.387950 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:38.387950 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:38.387950 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 07:20:38.387950 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 07:20:38.387950 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 07:20:38.387950 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 07:20:38.388967 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:38.388967 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:38.388967 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:38.388967 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:38.388967 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:38.388967 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:38.388967 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:38.388967 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:38.388967 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:38.388967 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:38.388967 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:38.388967 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:38.388967 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:38.388967 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:38.388967 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:38.388967 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:38.388967 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 07:20:38.390343 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 07:20:38.390343 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:20:38.390343 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:20:38.393516 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 252/517] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:20:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:38.394228 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86129780-a5a5-4101-8697-6b8b15848fcf None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:38.425509 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:38.487944 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e89ca77-6cda-41ec-a773-008361b4c058 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:38.494017 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86129780-a5a5-4101-8697-6b8b15848fcf None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:38.521149 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45a7ace-2a37-44c9-b336-76f9271fa0d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:38.521874 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45a7ace-2a37-44c9-b336-76f9271fa0d8 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:38.522151 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45a7ace-2a37-44c9-b336-76f9271fa0d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:38.522417 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45a7ace-2a37-44c9-b336-76f9271fa0d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:38.529193 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2421a4da-5a83-480b-b41f-8bb86c3abed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:38.529829 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2421a4da-5a83-480b-b41f-8bb86c3abed0 None admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:38.530692 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2421a4da-5a83-480b-b41f-8bb86c3abed0 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:38.531161 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2421a4da-5a83-480b-b41f-8bb86c3abed0 None admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:38.532182 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2421a4da-5a83-480b-b41f-8bb86c3abed0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:38.533994 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2421a4da-5a83-480b-b41f-8bb86c3abed0 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:38.570512 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:38.588538 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45a7ace-2a37-44c9-b336-76f9271fa0d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:38.590022 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45a7ace-2a37-44c9-b336-76f9271fa0d8 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:38.617433 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 253/518] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Sep 15 07:20:38 2022] POST /identity/v3/users => generated 247 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:20:38.618308 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2421a4da-5a83-480b-b41f-8bb86c3abed0 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:38.701931 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 266/519] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 15 07:20:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 07:20:38.703725 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45a7ace-2a37-44c9-b336-76f9271fa0d8 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:38.728148 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:38.747318 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2421a4da-5a83-480b-b41f-8bb86c3abed0 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:38.797801 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5a2129-7e50-499a-bd5f-8c0f49b93c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:38.797801 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5a2129-7e50-499a-bd5f-8c0f49b93c13 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:38.797801 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5a2129-7e50-499a-bd5f-8c0f49b93c13 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:38.797801 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5a2129-7e50-499a-bd5f-8c0f49b93c13 None admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:38.798371 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5a2129-7e50-499a-bd5f-8c0f49b93c13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:38.798751 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5a2129-7e50-499a-bd5f-8c0f49b93c13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:38.800947 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5a2129-7e50-499a-bd5f-8c0f49b93c13 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:38.808249 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:38.814058 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 254/520] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 15 07:20:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:38.814848 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5a2129-7e50-499a-bd5f-8c0f49b93c13 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:38.828001 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45a7ace-2a37-44c9-b336-76f9271fa0d8 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:38.869300 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013e5ef1-fbd3-4430-ad75-5788551c71cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:38.870097 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013e5ef1-fbd3-4430-ad75-5788551c71cb None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:38.870384 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013e5ef1-fbd3-4430-ad75-5788551c71cb None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:38.870658 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013e5ef1-fbd3-4430-ad75-5788551c71cb None admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:38.871308 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013e5ef1-fbd3-4430-ad75-5788551c71cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:38.871621 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013e5ef1-fbd3-4430-ad75-5788551c71cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:38.873017 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013e5ef1-fbd3-4430-ad75-5788551c71cb None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:38.885433 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 267/521] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 15 07:20:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:38.886372 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013e5ef1-fbd3-4430-ad75-5788551c71cb None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:38.943650 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5a2129-7e50-499a-bd5f-8c0f49b93c13 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:38.981068 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd25dd1b-84bf-48ea-85fa-47cc6a8ae165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:38.981820 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd25dd1b-84bf-48ea-85fa-47cc6a8ae165 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:38.982099 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd25dd1b-84bf-48ea-85fa-47cc6a8ae165 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:38.982372 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd25dd1b-84bf-48ea-85fa-47cc6a8ae165 None admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:38.983129 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd25dd1b-84bf-48ea-85fa-47cc6a8ae165 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:38.984174 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd25dd1b-84bf-48ea-85fa-47cc6a8ae165 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:38.985653 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd25dd1b-84bf-48ea-85fa-47cc6a8ae165 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:39.004094 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 255/522] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 15 07:20:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:39.004094 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd25dd1b-84bf-48ea-85fa-47cc6a8ae165 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:39.007718 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013e5ef1-fbd3-4430-ad75-5788551c71cb None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:39.011449 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:39.045417 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d26a44-22b6-4687-96c8-d53652361b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:39.046301 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d26a44-22b6-4687-96c8-d53652361b20 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:39.046581 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d26a44-22b6-4687-96c8-d53652361b20 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:39.046852 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d26a44-22b6-4687-96c8-d53652361b20 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:39.058754 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d26a44-22b6-4687-96c8-d53652361b20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:39.060651 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d26a44-22b6-4687-96c8-d53652361b20 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:39.063524 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:39.076923 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-76d26a44-22b6-4687-96c8-d53652361b20 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:20:39.076923 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:39.076923 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:39.076923 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:39.076923 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:39.076923 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:39.076923 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:39.076923 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:39.076923 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:39.076923 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:39.076923 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:39.076923 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:39.076923 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 07:20:39.076923 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 07:20:39.076923 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 07:20:39.076923 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 07:20:39.079157 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:39.079157 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:39.079157 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:39.079157 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:39.079157 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:39.079157 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:39.079157 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:39.079157 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:39.079157 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:39.079157 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:39.079157 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:39.079157 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:39.079157 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:39.079157 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:39.079157 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:39.079157 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:39.079157 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 07:20:39.081199 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 07:20:39.081199 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:20:39.081199 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:20:39.081199 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 268/523] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:20:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:39.081199 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d26a44-22b6-4687-96c8-d53652361b20 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:39.125254 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd25dd1b-84bf-48ea-85fa-47cc6a8ae165 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:39.161275 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8160bdbb-7fc8-4989-81cb-0fa7aff86b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:39.173416 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8160bdbb-7fc8-4989-81cb-0fa7aff86b9c None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:39.173416 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8160bdbb-7fc8-4989-81cb-0fa7aff86b9c None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:39.173416 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8160bdbb-7fc8-4989-81cb-0fa7aff86b9c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:39.173416 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8160bdbb-7fc8-4989-81cb-0fa7aff86b9c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:39.173416 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8160bdbb-7fc8-4989-81cb-0fa7aff86b9c None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:39.184389 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-8160bdbb-7fc8-4989-81cb-0fa7aff86b9c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 07:20:39.184389 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:39.184389 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:39.184389 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:39.184389 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:39.184389 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:39.184389 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:39.184389 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:39.184389 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:39.184389 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:39.184389 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:39.184389 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:39.184389 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 07:20:39.184389 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 07:20:39.184389 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 07:20:39.185535 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 07:20:39.185535 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:39.185535 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:39.185535 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:39.185535 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:39.185535 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:39.185535 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:39.185535 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:39.185535 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:39.185535 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:39.185535 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:39.185535 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:39.185535 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:39.185535 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:39.185535 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:39.185535 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:39.185535 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:39.186978 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 07:20:39.186978 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 07:20:39.186978 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 07:20:39.186978 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 07:20:39.186978 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 07:20:39.186978 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 07:20:39.186978 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 07:20:39.186978 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:20:39.186978 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 256/524] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 15 07:20:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:39.186978 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8160bdbb-7fc8-4989-81cb-0fa7aff86b9c None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:39.205216 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d26a44-22b6-4687-96c8-d53652361b20 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:39.242319 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14dc07d-5aa5-4348-9c24-3fb96afae9e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:39.242954 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14dc07d-5aa5-4348-9c24-3fb96afae9e3 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:39.243234 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14dc07d-5aa5-4348-9c24-3fb96afae9e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:39.243511 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14dc07d-5aa5-4348-9c24-3fb96afae9e3 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:39.252337 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14dc07d-5aa5-4348-9c24-3fb96afae9e3 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:39.253817 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14dc07d-5aa5-4348-9c24-3fb96afae9e3 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:39.262670 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:39.266699 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-b14dc07d-5aa5-4348-9c24-3fb96afae9e3 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 15 07:20:39.266699 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:39.266699 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:39.266699 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:39.266699 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:39.266699 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:39.266699 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:39.266699 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:39.266699 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:39.266699 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:39.266699 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:39.266699 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:39.266699 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 07:20:39.266699 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 07:20:39.266699 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 07:20:39.266699 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 07:20:39.267639 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:39.267639 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:39.267639 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 07:20:39.267639 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 07:20:39.267639 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 07:20:39.267639 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 07:20:39.267639 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:39.267639 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:39.267639 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:39.267639 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:39.267639 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:39.267639 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:39.267639 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:39.267639 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:39.267639 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:39.267639 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:39.267639 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:39.267639 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:39.268618 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:39.268618 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:39.268618 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 07:20:39.268618 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 07:20:39.268618 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 07:20:39.268618 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 07:20:39.268618 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 07:20:39.268618 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 07:20:39.268618 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 15 07:20:39.268618 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:20:39.268618 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 269/525] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Sep 15 07:20:39 2022] GET /identity/v3/users/neutron => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:39.268618 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14dc07d-5aa5-4348-9c24-3fb96afae9e3 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:39.302409 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8160bdbb-7fc8-4989-81cb-0fa7aff86b9c None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:39.328008 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:39.338593 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3cc086-295a-4ddf-b08e-9bd7c274d9a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:39.346867 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3cc086-295a-4ddf-b08e-9bd7c274d9a5 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:39.346867 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3cc086-295a-4ddf-b08e-9bd7c274d9a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:39.346867 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3cc086-295a-4ddf-b08e-9bd7c274d9a5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:39.346867 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3cc086-295a-4ddf-b08e-9bd7c274d9a5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:39.348118 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3cc086-295a-4ddf-b08e-9bd7c274d9a5 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:39.360907 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-ae3cc086-295a-4ddf-b08e-9bd7c274d9a5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 07:20:39.360907 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:39.360907 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:39.360907 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:39.360907 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:39.360907 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:39.360907 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:39.360907 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:39.360907 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:39.360907 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:39.360907 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:39.360907 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:39.360907 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 07:20:39.360907 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 07:20:39.360907 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 07:20:39.362000 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 07:20:39.362000 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:39.362000 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:39.362000 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:39.362000 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:39.362000 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:39.362000 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:39.362000 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:39.362000 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:39.362000 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:39.362000 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:39.362000 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:39.362000 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:39.362000 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:39.362000 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:39.362000 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:39.362000 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:39.367694 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 07:20:39.367694 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 07:20:39.367694 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 07:20:39.367694 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 07:20:39.367694 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 07:20:39.367694 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 07:20:39.367694 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 07:20:39.367694 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:20:39.367694 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 257/526] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 15 07:20:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:39.367694 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3cc086-295a-4ddf-b08e-9bd7c274d9a5 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:39.384277 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14dc07d-5aa5-4348-9c24-3fb96afae9e3 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:39.429887 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3497ef58-6b3e-467a-890e-85889da1270b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:39.430505 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3497ef58-6b3e-467a-890e-85889da1270b None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:39.430788 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3497ef58-6b3e-467a-890e-85889da1270b None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:39.431068 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3497ef58-6b3e-467a-890e-85889da1270b None admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:39.431712 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3497ef58-6b3e-467a-890e-85889da1270b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:39.432018 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3497ef58-6b3e-467a-890e-85889da1270b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:39.433723 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3497ef58-6b3e-467a-890e-85889da1270b None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:39.446956 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 270/527] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 15 07:20:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:39.447780 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3497ef58-6b3e-467a-890e-85889da1270b None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:39.486863 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3cc086-295a-4ddf-b08e-9bd7c274d9a5 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:39.514368 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:39.524179 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8ad072-a247-4b7d-b40e-afe2bbdada0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:39.524793 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ad072-a247-4b7d-b40e-afe2bbdada0b None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:39.525079 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ad072-a247-4b7d-b40e-afe2bbdada0b None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:39.525355 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ad072-a247-4b7d-b40e-afe2bbdada0b None admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:39.526097 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8ad072-a247-4b7d-b40e-afe2bbdada0b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:39.526397 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8ad072-a247-4b7d-b40e-afe2bbdada0b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:39.527841 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8ad072-a247-4b7d-b40e-afe2bbdada0b None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:39.551489 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 258/528] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 15 07:20:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:39.551489 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8ad072-a247-4b7d-b40e-afe2bbdada0b None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:39.565278 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3497ef58-6b3e-467a-890e-85889da1270b None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:39.582876 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:39.604408 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3d5575-521f-431e-a0be-6429aac6f212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:39.605134 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d5575-521f-431e-a0be-6429aac6f212 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:39.605414 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d5575-521f-431e-a0be-6429aac6f212 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:39.605716 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d5575-521f-431e-a0be-6429aac6f212 None admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:39.606438 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3d5575-521f-431e-a0be-6429aac6f212 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:39.606745 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3d5575-521f-431e-a0be-6429aac6f212 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:39.609144 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3d5575-521f-431e-a0be-6429aac6f212 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:39.630966 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 271/529] 10.222.0.42 () {56 vars in 1139 bytes} [Thu Sep 15 07:20:39 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:39.632137 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3d5575-521f-431e-a0be-6429aac6f212 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:39.663213 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8ad072-a247-4b7d-b40e-afe2bbdada0b None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:39.699033 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e46d36-1738-4a87-a4d5-67e93b3de880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:39.699633 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e46d36-1738-4a87-a4d5-67e93b3de880 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:39.699981 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e46d36-1738-4a87-a4d5-67e93b3de880 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:39.700186 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e46d36-1738-4a87-a4d5-67e93b3de880 None admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:39.700868 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e46d36-1738-4a87-a4d5-67e93b3de880 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:39.701286 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e46d36-1738-4a87-a4d5-67e93b3de880 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:39.702957 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e46d36-1738-4a87-a4d5-67e93b3de880 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:39.721384 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 259/530] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 15 07:20:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:39.723022 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e46d36-1738-4a87-a4d5-67e93b3de880 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:39.749360 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3d5575-521f-431e-a0be-6429aac6f212 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:39.784980 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e021cff4-0df3-4e27-823c-636f77d0d0da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:39.785532 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e021cff4-0df3-4e27-823c-636f77d0d0da None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:39.785855 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e021cff4-0df3-4e27-823c-636f77d0d0da None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:39.786125 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e021cff4-0df3-4e27-823c-636f77d0d0da None admin] PATH_INFO: `/v3/projects/service` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:39.787373 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:39.794249 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e021cff4-0df3-4e27-823c-636f77d0d0da None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:39.795707 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e021cff4-0df3-4e27-823c-636f77d0d0da None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:39.807939 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-e021cff4-0df3-4e27-823c-636f77d0d0da None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 07:20:39.807939 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:39.807939 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:39.807939 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:39.807939 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:39.807939 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:39.807939 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:39.807939 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:39.807939 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:39.807939 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:39.807939 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:39.807939 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:39.807939 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 07:20:39.807939 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 07:20:39.807939 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 07:20:39.809181 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 07:20:39.809181 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:39.809181 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:39.809181 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:39.809181 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:39.809181 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:39.809181 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:39.809181 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:39.809181 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:39.809181 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:39.809181 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:39.809181 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:39.809181 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:39.809181 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:39.809181 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:39.809181 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:39.809181 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:39.810076 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 07:20:39.810076 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 07:20:39.810076 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 07:20:39.810076 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 07:20:39.810076 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 07:20:39.810076 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 07:20:39.810076 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 07:20:39.810076 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:20:39.810076 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 272/531] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 15 07:20:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:39.810076 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e021cff4-0df3-4e27-823c-636f77d0d0da None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:39.841273 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:39.850787 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e46d36-1738-4a87-a4d5-67e93b3de880 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:39.890391 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e638a61-1993-414c-a2b7-ec3d990de2f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:39.891060 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e638a61-1993-414c-a2b7-ec3d990de2f9 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:39.891348 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e638a61-1993-414c-a2b7-ec3d990de2f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:39.891621 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e638a61-1993-414c-a2b7-ec3d990de2f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:39.892900 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e638a61-1993-414c-a2b7-ec3d990de2f9 None admin] RBAC: Adding query filter params (role.id=0e661b5a67664692b933e5da9fc2e488, scope.project.id=4e9987720930473da840579654311bc2, user.id=37438b032a1f46d3bdf7121ad0cffbc3) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:39.893214 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e638a61-1993-414c-a2b7-ec3d990de2f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:39.895202 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e638a61-1993-414c-a2b7-ec3d990de2f9 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:39.902393 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 260/532] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 15 07:20:39 2022] GET /identity/v3/role_assignments?user.id=37438b032a1f46d3bdf7121ad0cffbc3&scope.project.id=4e9987720930473da840579654311bc2&role.id=0e661b5a67664692b933e5da9fc2e488 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:39.909661 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e638a61-1993-414c-a2b7-ec3d990de2f9 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:39.921832 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e021cff4-0df3-4e27-823c-636f77d0d0da None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:39.972161 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acec0d3-42a4-49c6-aefc-8f3769e68cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:39.972937 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acec0d3-42a4-49c6-aefc-8f3769e68cb5 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:39.973404 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acec0d3-42a4-49c6-aefc-8f3769e68cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:39.973730 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acec0d3-42a4-49c6-aefc-8f3769e68cb5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:40.047459 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e638a61-1993-414c-a2b7-ec3d990de2f9 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:40.062580 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:40.071756 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acec0d3-42a4-49c6-aefc-8f3769e68cb5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:40.073654 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acec0d3-42a4-49c6-aefc-8f3769e68cb5 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:40.087219 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1861bbb9-894a-4f67-8fe8-80e9e32a0439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:40.088085 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1861bbb9-894a-4f67-8fe8-80e9e32a0439 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:40.088453 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1861bbb9-894a-4f67-8fe8-80e9e32a0439 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:40.088819 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1861bbb9-894a-4f67-8fe8-80e9e32a0439 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:40.089559 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1861bbb9-894a-4f67-8fe8-80e9e32a0439 None admin] RBAC: Adding query filter params (role.id=0e661b5a67664692b933e5da9fc2e488, scope.project.id=4e9987720930473da840579654311bc2, user.id=3ecf6642b7b84d3c9756bb41ab3349d3) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:40.089894 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1861bbb9-894a-4f67-8fe8-80e9e32a0439 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:40.091561 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1861bbb9-894a-4f67-8fe8-80e9e32a0439 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:40.105605 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 261/533] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 15 07:20:39 2022] GET /identity/v3/role_assignments?user.id=3ecf6642b7b84d3c9756bb41ab3349d3&scope.project.id=4e9987720930473da840579654311bc2&role.id=0e661b5a67664692b933e5da9fc2e488 => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:40.105605 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1861bbb9-894a-4f67-8fe8-80e9e32a0439 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:40.126360 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:40.216262 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1861bbb9-894a-4f67-8fe8-80e9e32a0439 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:40.273729 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 273/534] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 15 07:20:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 07:20:40.279523 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acec0d3-42a4-49c6-aefc-8f3769e68cb5 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:40.300148 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2da48d9-70a2-4481-8425-f8ad234fba27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:40.300686 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2da48d9-70a2-4481-8425-f8ad234fba27 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:40.300971 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2da48d9-70a2-4481-8425-f8ad234fba27 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:40.301238 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2da48d9-70a2-4481-8425-f8ad234fba27 None admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:40.301959 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2da48d9-70a2-4481-8425-f8ad234fba27 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:40.302261 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2da48d9-70a2-4481-8425-f8ad234fba27 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:40.303706 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2da48d9-70a2-4481-8425-f8ad234fba27 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:40.336669 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 262/535] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 15 07:20:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:40.348196 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2da48d9-70a2-4481-8425-f8ad234fba27 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:40.426640 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:40.478908 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acec0d3-42a4-49c6-aefc-8f3769e68cb5 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:40.541761 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:40.551325 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2da48d9-70a2-4481-8425-f8ad234fba27 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:40.552132 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987fede9-2c00-47ba-bfe7-fb81fe0f333e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:40.552949 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987fede9-2c00-47ba-bfe7-fb81fe0f333e None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:40.553388 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987fede9-2c00-47ba-bfe7-fb81fe0f333e None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:40.553940 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987fede9-2c00-47ba-bfe7-fb81fe0f333e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:40.573493 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987fede9-2c00-47ba-bfe7-fb81fe0f333e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:40.575271 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987fede9-2c00-47ba-bfe7-fb81fe0f333e None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:40.599807 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb2a82b-7cc3-4cfc-83f5-be756add02d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:40.600421 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb2a82b-7cc3-4cfc-83f5-be756add02d5 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:40.600705 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb2a82b-7cc3-4cfc-83f5-be756add02d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:40.600980 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb2a82b-7cc3-4cfc-83f5-be756add02d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:40.601755 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb2a82b-7cc3-4cfc-83f5-be756add02d5 None admin] RBAC: Adding query filter params (role.id=0e661b5a67664692b933e5da9fc2e488, scope.project.id=4e9987720930473da840579654311bc2, user.id=a080c7ab9b684abca6828067f2572c3b) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:40.602066 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb2a82b-7cc3-4cfc-83f5-be756add02d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:40.603498 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb2a82b-7cc3-4cfc-83f5-be756add02d5 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:40.605567 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-987fede9-2c00-47ba-bfe7-fb81fe0f333e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:20:40.605567 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:40.605567 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:40.605567 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:40.605567 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:40.605567 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:40.605567 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:40.605567 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:40.605567 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:40.605567 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:40.605567 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:40.605567 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:40.605567 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 07:20:40.605567 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 07:20:40.605567 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 07:20:40.605567 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 07:20:40.606432 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:40.606432 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:40.606432 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:40.606432 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:40.606432 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:40.606432 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:40.606432 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:40.606432 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:40.606432 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:40.606432 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:40.606432 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:40.606432 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:40.606432 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:40.606432 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:40.606432 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:40.606432 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:40.606432 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 07:20:40.607301 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 07:20:40.607301 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:20:40.607301 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:20:40.617635 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 263/536] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 15 07:20:40 2022] GET /identity/v3/role_assignments?user.id=a080c7ab9b684abca6828067f2572c3b&scope.project.id=4e9987720930473da840579654311bc2&role.id=0e661b5a67664692b933e5da9fc2e488 => generated 259 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:20:40.617635 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb2a82b-7cc3-4cfc-83f5-be756add02d5 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:40.618377 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 274/537] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:20:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 07:20:40.752186 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:40.755386 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb2a82b-7cc3-4cfc-83f5-be756add02d5 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:40.793407 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17277433-76a5-436a-b0b1-1ba8fd4527ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:40.794082 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17277433-76a5-436a-b0b1-1ba8fd4527ac None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:40.794368 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17277433-76a5-436a-b0b1-1ba8fd4527ac None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:40.794644 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17277433-76a5-436a-b0b1-1ba8fd4527ac None admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:40.795331 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17277433-76a5-436a-b0b1-1ba8fd4527ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:40.795751 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17277433-76a5-436a-b0b1-1ba8fd4527ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:40.797862 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17277433-76a5-436a-b0b1-1ba8fd4527ac None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:40.812260 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 264/538] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 15 07:20:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:40.817858 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987fede9-2c00-47ba-bfe7-fb81fe0f333e None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:40.927144 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c60083-cf21-43e7-8ae0-3da1f3acd34d None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:40.927608 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c60083-cf21-43e7-8ae0-3da1f3acd34d None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:40.927988 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c60083-cf21-43e7-8ae0-3da1f3acd34d None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:40.929878 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 265/539] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:20:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:20:40.935047 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae2ba1c-9cea-4379-a59f-d3eddee21d24 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:40.935334 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae2ba1c-9cea-4379-a59f-d3eddee21d24 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:40.935610 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae2ba1c-9cea-4379-a59f-d3eddee21d24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:40.972999 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987fede9-2c00-47ba-bfe7-fb81fe0f333e None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:40.981644 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-eae2ba1c-9cea-4379-a59f-d3eddee21d24 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:20:40.982855 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-eae2ba1c-9cea-4379-a59f-d3eddee21d24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:20:41.014066 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f660177-eb4d-425d-aeca-65ad5e3c5c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:41.015234 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f660177-eb4d-425d-aeca-65ad5e3c5c1b None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:41.015512 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f660177-eb4d-425d-aeca-65ad5e3c5c1b None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:41.015840 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f660177-eb4d-425d-aeca-65ad5e3c5c1b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:41.028595 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f660177-eb4d-425d-aeca-65ad5e3c5c1b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:41.030086 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f660177-eb4d-425d-aeca-65ad5e3c5c1b None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:41.051206 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-0f660177-eb4d-425d-aeca-65ad5e3c5c1b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 07:20:41.051206 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:41.051206 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:41.051206 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:41.051206 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:41.051206 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:41.051206 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:41.051206 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:41.051206 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:41.051206 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:41.051206 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:41.051206 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:41.051206 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 07:20:41.051206 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 07:20:41.051206 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 07:20:41.052831 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 07:20:41.052831 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:41.052831 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:41.052831 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:41.052831 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:41.052831 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:41.052831 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:41.052831 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:41.052831 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:41.052831 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:41.052831 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:41.052831 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:41.052831 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:41.052831 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:41.052831 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:41.052831 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:41.052831 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:41.054553 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 07:20:41.054553 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 07:20:41.054553 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 07:20:41.054553 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 07:20:41.054553 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 07:20:41.054553 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 07:20:41.054553 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 07:20:41.054553 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:20:41.054553 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 275/540] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 15 07:20:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:41.057987 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f660177-eb4d-425d-aeca-65ad5e3c5c1b None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:41.121767 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 266/541] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:20:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 07:20:41.146150 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:41.217123 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f660177-eb4d-425d-aeca-65ad5e3c5c1b None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:41.275292 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971a1a0c-e978-4345-b83e-391e95e1219f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:41.276721 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971a1a0c-e978-4345-b83e-391e95e1219f None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:41.277195 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971a1a0c-e978-4345-b83e-391e95e1219f None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:41.277696 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971a1a0c-e978-4345-b83e-391e95e1219f None admin] PATH_INFO: `/v3/projects` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:41.278645 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971a1a0c-e978-4345-b83e-391e95e1219f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:41.279147 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971a1a0c-e978-4345-b83e-391e95e1219f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:41.281350 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971a1a0c-e978-4345-b83e-391e95e1219f None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:41.314241 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 276/542] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 15 07:20:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:41.326735 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae2ba1c-9cea-4379-a59f-d3eddee21d24 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:41.460942 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:41.471054 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae2ba1c-9cea-4379-a59f-d3eddee21d24 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:41.491505 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971a1a0c-e978-4345-b83e-391e95e1219f None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:41.526518 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744f9fcf-0c80-46d7-a815-f67daf1a3fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:41.527471 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744f9fcf-0c80-46d7-a815-f67daf1a3fd5 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:41.528117 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744f9fcf-0c80-46d7-a815-f67daf1a3fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:41.529552 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744f9fcf-0c80-46d7-a815-f67daf1a3fd5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:41.531072 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744f9fcf-0c80-46d7-a815-f67daf1a3fd5 None admin] RBAC: Adding query filter params (role.id=0e661b5a67664692b933e5da9fc2e488, scope.project.id=4e9987720930473da840579654311bc2, user.id=5315996432f44bffa3460b2744c10f0a) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:41.531792 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744f9fcf-0c80-46d7-a815-f67daf1a3fd5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:41.535370 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744f9fcf-0c80-46d7-a815-f67daf1a3fd5 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:41.550531 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 267/543] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 15 07:20:41 2022] GET /identity/v3/role_assignments?user.id=5315996432f44bffa3460b2744c10f0a&scope.project.id=4e9987720930473da840579654311bc2&role.id=0e661b5a67664692b933e5da9fc2e488 => generated 259 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:41.652842 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971a1a0c-e978-4345-b83e-391e95e1219f None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:41.656716 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:41.706598 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0cfd12-5797-466e-bf64-61cbc568c9ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:41.707637 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0cfd12-5797-466e-bf64-61cbc568c9ac None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:41.708134 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0cfd12-5797-466e-bf64-61cbc568c9ac None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:41.708745 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0cfd12-5797-466e-bf64-61cbc568c9ac None admin] PATH_INFO: `/v3/roles/service` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:41.730463 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0cfd12-5797-466e-bf64-61cbc568c9ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:20:41.736832 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-4f0cfd12-5797-466e-bf64-61cbc568c9ac None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 07:20:41.736832 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:41.736832 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:41.736832 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:41.736832 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:41.736832 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:41.736832 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:41.736832 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:41.736832 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:41.736832 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:41.736832 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:41.736832 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:41.736832 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:20:41.736832 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:20:41.736832 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:20:41.736832 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:20:41.737792 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:20:41.737792 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:20:41.737792 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:20:41.737792 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:20:41.737792 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:41.737792 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:41.737792 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:41.737792 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:41.737792 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:41.737792 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:41.737792 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:41.737792 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:41.737792 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:41.737792 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:41.737792 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:41.737792 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:41.738567 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:41.738567 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:41.738567 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:41.738567 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:41.738567 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:20:41.738567 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:20:41.738567 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:20:41.738567 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:20:41.738567 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:20:41.738567 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:20:41.738567 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 07:20:41.738567 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:20:41.740921 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 277/544] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Sep 15 07:20:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 07:20:41.746901 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744f9fcf-0c80-46d7-a815-f67daf1a3fd5 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:41.885062 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744f9fcf-0c80-46d7-a815-f67daf1a3fd5 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:41.919676 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a883e0-404e-4341-9e00-45cf586c5184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:41.920283 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a883e0-404e-4341-9e00-45cf586c5184 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:41.920564 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a883e0-404e-4341-9e00-45cf586c5184 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:41.921363 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a883e0-404e-4341-9e00-45cf586c5184 None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:41.922067 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a883e0-404e-4341-9e00-45cf586c5184 None admin] RBAC: Adding query filter params (name=service) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:41.922370 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a883e0-404e-4341-9e00-45cf586c5184 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:41.923744 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a883e0-404e-4341-9e00-45cf586c5184 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:41.932150 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 268/545] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Sep 15 07:20:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:41.937771 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0cfd12-5797-466e-bf64-61cbc568c9ac None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:42.073246 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0cfd12-5797-466e-bf64-61cbc568c9ac None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:42.109756 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814bc6fe-fd5a-44bc-bdf8-d0c721356c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:42.110492 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814bc6fe-fd5a-44bc-bdf8-d0c721356c2b None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:42.110777 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814bc6fe-fd5a-44bc-bdf8-d0c721356c2b None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:42.111054 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814bc6fe-fd5a-44bc-bdf8-d0c721356c2b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:42.122269 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814bc6fe-fd5a-44bc-bdf8-d0c721356c2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:42.123771 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814bc6fe-fd5a-44bc-bdf8-d0c721356c2b None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:42.138364 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-814bc6fe-fd5a-44bc-bdf8-d0c721356c2b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:20:42.138364 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:42.138364 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:42.138364 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:42.138364 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:42.138364 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:42.138364 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:42.138364 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:42.138364 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:42.138364 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:42.138364 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:42.138364 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:42.138364 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 07:20:42.138364 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 07:20:42.138364 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 07:20:42.138364 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 07:20:42.139360 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:42.139360 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:42.139360 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:42.139360 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:42.139360 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:42.139360 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:42.139360 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:42.139360 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:42.139360 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:42.139360 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:42.139360 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:42.139360 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:42.139360 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:42.139360 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:42.139360 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:42.139360 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:42.139360 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 07:20:42.140252 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 07:20:42.140252 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:20:42.140252 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:20:42.140252 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 278/546] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:20:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 07:20:42.145604 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a883e0-404e-4341-9e00-45cf586c5184 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:42.166669 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:42.337129 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a883e0-404e-4341-9e00-45cf586c5184 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:42.377157 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a26ff3-da0b-4da9-8a64-5508d23b8c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:42.377817 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a26ff3-da0b-4da9-8a64-5508d23b8c48 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:42.378331 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a26ff3-da0b-4da9-8a64-5508d23b8c48 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:42.378331 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a26ff3-da0b-4da9-8a64-5508d23b8c48 None admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:42.378997 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a26ff3-da0b-4da9-8a64-5508d23b8c48 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:42.379295 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a26ff3-da0b-4da9-8a64-5508d23b8c48 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:42.380667 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a26ff3-da0b-4da9-8a64-5508d23b8c48 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:42.398846 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 269/547] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 15 07:20:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:42.404008 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814bc6fe-fd5a-44bc-bdf8-d0c721356c2b None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:42.437947 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:42.572384 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814bc6fe-fd5a-44bc-bdf8-d0c721356c2b None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:42.582650 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680f8a5d-2bd2-4a8c-a4ba-24abbc785261 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:42.583138 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680f8a5d-2bd2-4a8c-a4ba-24abbc785261 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:42.583408 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680f8a5d-2bd2-4a8c-a4ba-24abbc785261 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:42.585096 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 270/548] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:20:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 07:20:42.591222 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00097600-7f64-4f36-ab34-f71e80fbcb92 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:42.591428 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00097600-7f64-4f36-ab34-f71e80fbcb92 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:42.591706 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00097600-7f64-4f36-ab34-f71e80fbcb92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:42.620732 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-00097600-7f64-4f36-ab34-f71e80fbcb92 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:20:42.621951 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-00097600-7f64-4f36-ab34-f71e80fbcb92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:20:42.638573 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e98276-0e55-4259-87db-31f8760e5e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:42.639219 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e98276-0e55-4259-87db-31f8760e5e1b None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:42.639497 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e98276-0e55-4259-87db-31f8760e5e1b None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:42.640316 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e98276-0e55-4259-87db-31f8760e5e1b None admin] PATH_INFO: `/v3/users/swift` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:42.651451 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e98276-0e55-4259-87db-31f8760e5e1b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:42.652925 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e98276-0e55-4259-87db-31f8760e5e1b None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:42.673562 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-b7e98276-0e55-4259-87db-31f8760e5e1b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 15 07:20:42.673562 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:42.673562 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:42.673562 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:42.673562 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:42.673562 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:42.673562 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:42.673562 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:42.673562 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:42.673562 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:42.673562 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:42.673562 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:42.673562 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 07:20:42.673562 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 07:20:42.673562 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 07:20:42.673562 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 07:20:42.674529 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:42.674529 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:42.674529 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 07:20:42.674529 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 07:20:42.674529 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 07:20:42.674529 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 07:20:42.674529 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:42.674529 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:42.674529 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:42.674529 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:42.674529 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:42.674529 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:42.674529 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:42.674529 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:42.674529 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:42.674529 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:42.674529 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:42.674529 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:42.675427 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:42.675427 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:42.675427 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 07:20:42.675427 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 07:20:42.675427 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 07:20:42.675427 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 07:20:42.675427 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 07:20:42.675427 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 07:20:42.675427 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 15 07:20:42.675427 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:20:42.676419 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 279/549] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 15 07:20:42 2022] GET /identity/v3/users/swift => generated 83 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:42.684080 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e98276-0e55-4259-87db-31f8760e5e1b None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:42.772426 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 271/550] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:20:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 07:20:42.837959 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:42.862922 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e98276-0e55-4259-87db-31f8760e5e1b None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:42.878941 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02061f79-61f1-4958-9436-4b365a81cda6 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:42.879600 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02061f79-61f1-4958-9436-4b365a81cda6 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:42.880305 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02061f79-61f1-4958-9436-4b365a81cda6 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:42.882261 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 272/551] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:20:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 07:20:42.887636 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79815e6-3495-4f97-84c1-03741e1adcd7 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:42.888044 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79815e6-3495-4f97-84c1-03741e1adcd7 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:42.888431 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79815e6-3495-4f97-84c1-03741e1adcd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:42.896360 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68efce06-7c1f-4d7f-ba5a-295909cbbd15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:42.897206 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68efce06-7c1f-4d7f-ba5a-295909cbbd15 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:42.897665 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68efce06-7c1f-4d7f-ba5a-295909cbbd15 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:42.898425 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68efce06-7c1f-4d7f-ba5a-295909cbbd15 None admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:42.899302 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68efce06-7c1f-4d7f-ba5a-295909cbbd15 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:42.899752 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68efce06-7c1f-4d7f-ba5a-295909cbbd15 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:42.901337 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68efce06-7c1f-4d7f-ba5a-295909cbbd15 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:42.904474 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:42.911009 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-a79815e6-3495-4f97-84c1-03741e1adcd7 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:20:42.914555 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-a79815e6-3495-4f97-84c1-03741e1adcd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:20:42.920607 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 280/552] 10.222.0.42 () {56 vars in 1135 bytes} [Thu Sep 15 07:20:42 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:20:42.927573 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68efce06-7c1f-4d7f-ba5a-295909cbbd15 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:43.040188 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 273/553] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:20:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 07:20:43.080504 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68efce06-7c1f-4d7f-ba5a-295909cbbd15 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:43.138381 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6485c61b-ca8a-49a1-a6da-38020c28789d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:43.138913 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6485c61b-ca8a-49a1-a6da-38020c28789d None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:43.139200 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6485c61b-ca8a-49a1-a6da-38020c28789d None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:43.139474 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6485c61b-ca8a-49a1-a6da-38020c28789d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:43.143902 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:43.210925 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79815e6-3495-4f97-84c1-03741e1adcd7 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:43.238299 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6485c61b-ca8a-49a1-a6da-38020c28789d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:43.239773 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6485c61b-ca8a-49a1-a6da-38020c28789d None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:43.331552 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79815e6-3495-4f97-84c1-03741e1adcd7 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:43.369106 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f122f3-1d60-4e69-8677-4b7b43415766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:43.369710 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f122f3-1d60-4e69-8677-4b7b43415766 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:43.369996 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f122f3-1d60-4e69-8677-4b7b43415766 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:43.370268 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f122f3-1d60-4e69-8677-4b7b43415766 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:43.373656 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:43.383628 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f122f3-1d60-4e69-8677-4b7b43415766 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:20:43.390066 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-42f122f3-1d60-4e69-8677-4b7b43415766 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 07:20:43.390066 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:43.390066 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:43.390066 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:43.390066 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:43.390066 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:43.390066 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:43.390066 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:43.390066 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:43.390066 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:43.390066 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:43.390066 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:43.390066 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:20:43.390066 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:20:43.390066 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:20:43.390066 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:20:43.391055 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:20:43.391055 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:20:43.391055 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:20:43.391055 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:20:43.391055 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:43.391055 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:43.391055 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:43.391055 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:43.391055 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:43.391055 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:43.391055 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:43.391055 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:43.391055 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:43.391055 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:43.391055 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:43.391055 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:43.391894 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:43.391894 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:43.391894 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:43.391894 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:43.391894 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:20:43.391894 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:20:43.391894 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:20:43.391894 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:20:43.391894 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:20:43.391894 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:20:43.391894 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 07:20:43.391894 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:20:43.394320 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 274/554] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Sep 15 07:20:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 07:20:43.395531 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f122f3-1d60-4e69-8677-4b7b43415766 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:43.414391 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 281/555] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 15 07:20:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 15 07:20:43.415230 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6485c61b-ca8a-49a1-a6da-38020c28789d None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:43.453830 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:43.517027 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f122f3-1d60-4e69-8677-4b7b43415766 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:43.522080 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6485c61b-ca8a-49a1-a6da-38020c28789d None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:43.551700 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9fb737-0c67-440b-ad15-8f2d772fdaee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:43.552327 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9fb737-0c67-440b-ad15-8f2d772fdaee None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:43.552604 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9fb737-0c67-440b-ad15-8f2d772fdaee None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:43.552875 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9fb737-0c67-440b-ad15-8f2d772fdaee None admin] PATH_INFO: `/v3/roles/service` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:43.557479 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c64b3b-5777-47d4-81db-cd0e4c13f292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:43.558115 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c64b3b-5777-47d4-81db-cd0e4c13f292 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:43.558400 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c64b3b-5777-47d4-81db-cd0e4c13f292 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:43.558932 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c64b3b-5777-47d4-81db-cd0e4c13f292 None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:43.559596 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c64b3b-5777-47d4-81db-cd0e4c13f292 None admin] RBAC: Adding query filter params (name=service) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:43.559896 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c64b3b-5777-47d4-81db-cd0e4c13f292 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:43.561288 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c64b3b-5777-47d4-81db-cd0e4c13f292 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:43.563642 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9fb737-0c67-440b-ad15-8f2d772fdaee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:20:43.568490 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-1b9fb737-0c67-440b-ad15-8f2d772fdaee None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 07:20:43.568490 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:43.568490 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:43.568490 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:43.568490 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:43.568490 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:43.568490 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:43.568490 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:43.568490 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:43.568490 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:43.568490 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:43.568490 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:43.568490 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:20:43.568490 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:20:43.568490 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:20:43.568490 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:20:43.569465 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:20:43.569465 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:20:43.569465 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:20:43.569465 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:20:43.569465 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:43.569465 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:43.569465 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:43.569465 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:43.569465 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:43.569465 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:43.569465 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:43.569465 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:43.569465 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:43.569465 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:43.569465 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:43.569465 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:43.570942 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:43.570942 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:43.570942 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:43.570942 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:43.570942 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:20:43.570942 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:20:43.570942 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:20:43.570942 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:20:43.570942 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:20:43.570942 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:20:43.570942 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 07:20:43.570942 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:20:43.571584 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 282/556] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Sep 15 07:20:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:43.571584 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1d90f6-2658-4d75-bdbf-9e4944376ec2 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:43.571584 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1d90f6-2658-4d75-bdbf-9e4944376ec2 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:43.571584 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1d90f6-2658-4d75-bdbf-9e4944376ec2 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:43.577170 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 283/557] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:20:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:20:43.577170 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1d90f6-2658-4d75-bdbf-9e4944376ec2 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:43.577396 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 275/558] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Sep 15 07:20:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:43.577396 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9fb737-0c67-440b-ad15-8f2d772fdaee None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:43.638309 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:43.686152 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:43.690692 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9fb737-0c67-440b-ad15-8f2d772fdaee None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:43.693833 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1d90f6-2658-4d75-bdbf-9e4944376ec2 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:43.726547 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dca5f3f-7eb0-4fc2-8c31-ed269b9f6971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:43.727195 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dca5f3f-7eb0-4fc2-8c31-ed269b9f6971 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:43.727460 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dca5f3f-7eb0-4fc2-8c31-ed269b9f6971 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:43.727733 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dca5f3f-7eb0-4fc2-8c31-ed269b9f6971 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:43.731170 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f057f878-fff5-4774-b02d-7c91ad32fe6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:43.731969 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f057f878-fff5-4774-b02d-7c91ad32fe6e None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:43.732433 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f057f878-fff5-4774-b02d-7c91ad32fe6e None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:43.732854 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f057f878-fff5-4774-b02d-7c91ad32fe6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:43.739747 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dca5f3f-7eb0-4fc2-8c31-ed269b9f6971 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:43.740855 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dca5f3f-7eb0-4fc2-8c31-ed269b9f6971 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:43.748885 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f057f878-fff5-4774-b02d-7c91ad32fe6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:43.751064 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f057f878-fff5-4774-b02d-7c91ad32fe6e None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:43.755558 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-1dca5f3f-7eb0-4fc2-8c31-ed269b9f6971 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:20:43.755558 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:43.755558 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:43.755558 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:43.755558 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:43.755558 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:43.755558 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:43.755558 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:43.755558 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:43.755558 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:43.755558 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:43.755558 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:43.755558 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 07:20:43.755558 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 07:20:43.755558 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 07:20:43.755558 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 07:20:43.756722 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:43.756722 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:43.756722 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:43.756722 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:43.756722 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:43.756722 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:43.756722 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:43.756722 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:43.756722 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:43.756722 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:43.756722 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:43.756722 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:43.756722 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:43.756722 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:43.756722 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:43.756722 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:43.756722 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 07:20:43.757628 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 07:20:43.757628 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:20:43.757628 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:20:43.757628 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 276/559] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:20:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:43.758238 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047da4f2-f10f-4e63-bd7b-ab9464991b12 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:43.758522 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047da4f2-f10f-4e63-bd7b-ab9464991b12 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:43.758794 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047da4f2-f10f-4e63-bd7b-ab9464991b12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:43.770353 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-f057f878-fff5-4774-b02d-7c91ad32fe6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:20:43.770353 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:43.770353 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:43.770353 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:43.770353 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:43.770353 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:43.770353 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:43.770353 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:43.770353 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:43.770353 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:43.770353 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:43.770353 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:43.770353 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 07:20:43.770353 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 07:20:43.770353 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 07:20:43.770353 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 07:20:43.771493 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:43.771493 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:43.771493 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:43.771493 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:43.771493 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:43.771493 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:43.771493 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:43.771493 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:43.771493 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:43.771493 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:43.771493 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:43.771493 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:43.771493 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:43.771493 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:43.771493 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:43.771493 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:43.771493 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 07:20:43.772574 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 07:20:43.772574 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:20:43.772574 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:20:43.774012 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 284/560] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:20:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:43.774790 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f057f878-fff5-4774-b02d-7c91ad32fe6e None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:43.783591 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-047da4f2-f10f-4e63-bd7b-ab9464991b12 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:20:43.784847 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-047da4f2-f10f-4e63-bd7b-ab9464991b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:20:43.859901 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 277/561] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:20:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 07:20:43.860749 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047da4f2-f10f-4e63-bd7b-ab9464991b12 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:43.891320 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f057f878-fff5-4774-b02d-7c91ad32fe6e None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:43.906292 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:43.935156 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7428e0b-3b5b-4b16-8ea6-c3a713cb0018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:43.935863 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7428e0b-3b5b-4b16-8ea6-c3a713cb0018 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:43.936329 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7428e0b-3b5b-4b16-8ea6-c3a713cb0018 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:43.936732 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7428e0b-3b5b-4b16-8ea6-c3a713cb0018 None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:43.944076 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7428e0b-3b5b-4b16-8ea6-c3a713cb0018 None admin] RBAC: Adding query filter params (name=service) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:43.944076 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7428e0b-3b5b-4b16-8ea6-c3a713cb0018 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:43.951824 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7428e0b-3b5b-4b16-8ea6-c3a713cb0018 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:43.969324 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 285/562] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Sep 15 07:20:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:43.970257 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7428e0b-3b5b-4b16-8ea6-c3a713cb0018 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:43.982180 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047da4f2-f10f-4e63-bd7b-ab9464991b12 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:43.986679 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:44.019691 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6680f36f-e0b2-490d-8107-3e0c98d3c962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:44.020977 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6680f36f-e0b2-490d-8107-3e0c98d3c962 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:44.020977 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6680f36f-e0b2-490d-8107-3e0c98d3c962 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:44.020977 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6680f36f-e0b2-490d-8107-3e0c98d3c962 None admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:44.021652 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6680f36f-e0b2-490d-8107-3e0c98d3c962 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:44.021959 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6680f36f-e0b2-490d-8107-3e0c98d3c962 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:44.023321 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6680f36f-e0b2-490d-8107-3e0c98d3c962 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:44.036053 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 278/563] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 15 07:20:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:44.037141 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6680f36f-e0b2-490d-8107-3e0c98d3c962 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:44.099076 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7428e0b-3b5b-4b16-8ea6-c3a713cb0018 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:44.138749 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3303bb69-c108-4363-81e8-a1f862d96826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:44.138970 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3303bb69-c108-4363-81e8-a1f862d96826 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:44.139214 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3303bb69-c108-4363-81e8-a1f862d96826 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:44.139468 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3303bb69-c108-4363-81e8-a1f862d96826 None admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:44.140969 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3303bb69-c108-4363-81e8-a1f862d96826 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:44.141350 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3303bb69-c108-4363-81e8-a1f862d96826 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:44.143136 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3303bb69-c108-4363-81e8-a1f862d96826 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:44.155245 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 286/564] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 15 07:20:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:44.156243 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3303bb69-c108-4363-81e8-a1f862d96826 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:44.169562 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:44.176111 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6680f36f-e0b2-490d-8107-3e0c98d3c962 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:44.220240 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931a5032-6d50-43f4-9e80-fce4133137af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:44.220799 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931a5032-6d50-43f4-9e80-fce4133137af None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:44.221082 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931a5032-6d50-43f4-9e80-fce4133137af None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:44.221365 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931a5032-6d50-43f4-9e80-fce4133137af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:44.228775 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:44.232362 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931a5032-6d50-43f4-9e80-fce4133137af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:44.233915 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931a5032-6d50-43f4-9e80-fce4133137af None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:44.248501 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-931a5032-6d50-43f4-9e80-fce4133137af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:20:44.248501 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:44.248501 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:44.248501 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:44.248501 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:44.248501 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:44.248501 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:44.248501 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:44.248501 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:44.248501 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:44.248501 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:44.248501 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:44.248501 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 07:20:44.248501 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 07:20:44.248501 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 07:20:44.248501 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 07:20:44.250559 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:44.250559 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:44.250559 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:44.250559 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:44.250559 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:44.250559 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:44.250559 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:44.250559 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:44.250559 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:44.250559 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:44.250559 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:44.250559 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:44.250559 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:44.250559 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:44.250559 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:44.250559 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:44.250559 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 07:20:44.252062 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 07:20:44.252062 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:20:44.252062 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:20:44.252062 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 279/565] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:20:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:44.252062 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931a5032-6d50-43f4-9e80-fce4133137af None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:44.261218 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3303bb69-c108-4363-81e8-a1f862d96826 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:44.295622 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be053cdf-c824-47c1-8442-98ce5290c519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:44.296277 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be053cdf-c824-47c1-8442-98ce5290c519 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:44.296563 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be053cdf-c824-47c1-8442-98ce5290c519 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:44.296830 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be053cdf-c824-47c1-8442-98ce5290c519 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:44.303655 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be053cdf-c824-47c1-8442-98ce5290c519 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:44.305094 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be053cdf-c824-47c1-8442-98ce5290c519 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:44.317184 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-be053cdf-c824-47c1-8442-98ce5290c519 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 15 07:20:44.317184 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:44.317184 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:44.317184 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:44.317184 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:44.317184 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:44.317184 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:44.317184 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:44.317184 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:44.317184 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:44.317184 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:44.317184 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:44.317184 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 07:20:44.317184 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 07:20:44.317184 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 07:20:44.317184 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 07:20:44.318172 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:44.318172 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:44.318172 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 07:20:44.318172 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 07:20:44.318172 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 07:20:44.318172 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 07:20:44.318172 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:44.318172 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:44.318172 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:44.318172 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:44.318172 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:44.318172 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:44.318172 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:44.318172 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:44.318172 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:44.318172 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:44.318172 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:44.318172 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:44.319538 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:44.319538 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:44.319538 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 07:20:44.319538 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 07:20:44.319538 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 07:20:44.319538 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 07:20:44.319538 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 07:20:44.319538 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 07:20:44.319538 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 15 07:20:44.319538 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:20:44.319538 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 287/566] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 15 07:20:44 2022] GET /identity/v3/users/glance => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:44.319538 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be053cdf-c824-47c1-8442-98ce5290c519 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:44.354598 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931a5032-6d50-43f4-9e80-fce4133137af None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:44.388010 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e8e72f-8215-4118-a91a-474e8ce62f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:44.388639 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e8e72f-8215-4118-a91a-474e8ce62f9a None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:44.388924 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e8e72f-8215-4118-a91a-474e8ce62f9a None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:44.389201 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e8e72f-8215-4118-a91a-474e8ce62f9a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:44.399204 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e8e72f-8215-4118-a91a-474e8ce62f9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:20:44.404422 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-94e8e72f-8215-4118-a91a-474e8ce62f9a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 07:20:44.404422 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:44.404422 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:44.404422 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:44.404422 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:44.404422 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:44.404422 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:44.404422 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:44.404422 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:44.404422 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:44.404422 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:44.404422 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:44.404422 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:20:44.404422 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:20:44.404422 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:20:44.404422 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:20:44.406011 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:20:44.406011 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:20:44.406011 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:20:44.406011 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:20:44.406011 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:44.406011 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:44.406011 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:44.406011 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:44.406011 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:44.406011 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:44.406011 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:44.406011 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:44.406011 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:44.406011 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:44.406011 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:44.406011 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:44.406953 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:44.406953 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:44.406953 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:44.406953 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:44.406953 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:20:44.406953 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:20:44.406953 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:20:44.406953 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:20:44.406953 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:20:44.406953 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:20:44.406953 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 07:20:44.406953 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:20:44.406953 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 280/567] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Sep 15 07:20:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:44.407690 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb39b4d-903b-4bc7-951d-93dec19e7a9b None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:44.407690 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb39b4d-903b-4bc7-951d-93dec19e7a9b None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:44.407813 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb39b4d-903b-4bc7-951d-93dec19e7a9b None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:44.409162 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 281/568] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:20:44 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:20:44.411509 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb39b4d-903b-4bc7-951d-93dec19e7a9b None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:44.422608 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be053cdf-c824-47c1-8442-98ce5290c519 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:44.423191 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:44.451740 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:44.457533 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef63f072-c29e-4d33-84ea-d675a613ec96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:44.458198 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef63f072-c29e-4d33-84ea-d675a613ec96 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:44.458483 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef63f072-c29e-4d33-84ea-d675a613ec96 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:44.458758 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef63f072-c29e-4d33-84ea-d675a613ec96 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:44.465294 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef63f072-c29e-4d33-84ea-d675a613ec96 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:44.466790 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef63f072-c29e-4d33-84ea-d675a613ec96 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:44.479329 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-ef63f072-c29e-4d33-84ea-d675a613ec96 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 07:20:44.479329 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:44.479329 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:44.479329 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:44.479329 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:44.479329 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:44.479329 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:44.479329 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:44.479329 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:44.479329 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:44.479329 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:44.479329 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:44.479329 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 07:20:44.479329 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 07:20:44.479329 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 07:20:44.480196 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 07:20:44.480196 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:44.480196 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:44.480196 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:44.480196 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:44.480196 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:44.480196 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:44.480196 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:44.480196 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:44.480196 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:44.480196 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:44.480196 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:44.480196 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:44.480196 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:44.480196 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:44.480196 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:44.480196 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:44.481127 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 07:20:44.481127 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 07:20:44.481127 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 07:20:44.481127 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 07:20:44.481127 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 07:20:44.481127 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 07:20:44.481127 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 07:20:44.481127 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:20:44.481127 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 288/569] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 15 07:20:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:44.481127 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef63f072-c29e-4d33-84ea-d675a613ec96 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:44.516738 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb39b4d-903b-4bc7-951d-93dec19e7a9b None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:44.550175 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9ffc2d-66c5-4129-ae2c-4a5d0cf8aac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:44.550770 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9ffc2d-66c5-4129-ae2c-4a5d0cf8aac9 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:44.551051 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9ffc2d-66c5-4129-ae2c-4a5d0cf8aac9 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:44.551319 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9ffc2d-66c5-4129-ae2c-4a5d0cf8aac9 None admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:44.551951 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9ffc2d-66c5-4129-ae2c-4a5d0cf8aac9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:44.552253 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9ffc2d-66c5-4129-ae2c-4a5d0cf8aac9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:44.553631 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9ffc2d-66c5-4129-ae2c-4a5d0cf8aac9 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:44.565934 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 282/570] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 15 07:20:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:44.566746 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9ffc2d-66c5-4129-ae2c-4a5d0cf8aac9 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:44.584749 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef63f072-c29e-4d33-84ea-d675a613ec96 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:44.618167 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93458df5-8538-446f-ad6d-9e837f968011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:44.618777 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93458df5-8538-446f-ad6d-9e837f968011 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:44.619062 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93458df5-8538-446f-ad6d-9e837f968011 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:44.619337 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93458df5-8538-446f-ad6d-9e837f968011 None admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:44.620063 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93458df5-8538-446f-ad6d-9e837f968011 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:44.620375 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93458df5-8538-446f-ad6d-9e837f968011 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:44.621835 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93458df5-8538-446f-ad6d-9e837f968011 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:44.638405 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:44.639569 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 289/571] 10.222.0.42 () {56 vars in 1137 bytes} [Thu Sep 15 07:20:44 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:44.641167 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e4bceb-9e4b-4fb8-9f15-29ca57fb7da1 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:44.641450 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e4bceb-9e4b-4fb8-9f15-29ca57fb7da1 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:44.641762 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e4bceb-9e4b-4fb8-9f15-29ca57fb7da1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:44.659284 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-82e4bceb-9e4b-4fb8-9f15-29ca57fb7da1 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:20:44.660477 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-82e4bceb-9e4b-4fb8-9f15-29ca57fb7da1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:20:44.668904 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9ffc2d-66c5-4129-ae2c-4a5d0cf8aac9 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:44.704917 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6715337f-d031-4d11-bafb-fec43b7774ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:44.705530 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6715337f-d031-4d11-bafb-fec43b7774ab None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:44.705688 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:44.705839 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6715337f-d031-4d11-bafb-fec43b7774ab None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:44.706485 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6715337f-d031-4d11-bafb-fec43b7774ab None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:44.707152 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6715337f-d031-4d11-bafb-fec43b7774ab None admin] RBAC: Adding query filter params (name=service) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:44.707463 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6715337f-d031-4d11-bafb-fec43b7774ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:44.708857 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6715337f-d031-4d11-bafb-fec43b7774ab None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:44.716754 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 283/572] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Sep 15 07:20:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:44.717669 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6715337f-d031-4d11-bafb-fec43b7774ab None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:44.728841 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 290/573] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:20:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 07:20:44.729752 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e4bceb-9e4b-4fb8-9f15-29ca57fb7da1 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:44.831447 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6715337f-d031-4d11-bafb-fec43b7774ab None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:44.836996 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e4bceb-9e4b-4fb8-9f15-29ca57fb7da1 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:44.862565 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:44.868041 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea738358-99a6-487e-8b49-7f86f4a172db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:44.868650 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea738358-99a6-487e-8b49-7f86f4a172db None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:44.868935 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea738358-99a6-487e-8b49-7f86f4a172db None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:44.869220 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea738358-99a6-487e-8b49-7f86f4a172db None admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:44.870178 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea738358-99a6-487e-8b49-7f86f4a172db None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:44.874846 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea738358-99a6-487e-8b49-7f86f4a172db None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:44.874846 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea738358-99a6-487e-8b49-7f86f4a172db None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:44.879260 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2b43bf-2878-44ee-812c-b614d7459aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:44.880446 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2b43bf-2878-44ee-812c-b614d7459aff None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:44.880732 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2b43bf-2878-44ee-812c-b614d7459aff None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:44.881005 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2b43bf-2878-44ee-812c-b614d7459aff None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:44.884792 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 284/574] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 15 07:20:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:44.889771 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea738358-99a6-487e-8b49-7f86f4a172db None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:44.895728 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2b43bf-2878-44ee-812c-b614d7459aff None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:44.897188 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2b43bf-2878-44ee-812c-b614d7459aff None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:44.908958 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-cb2b43bf-2878-44ee-812c-b614d7459aff None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 15 07:20:44.908958 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:44.908958 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:44.908958 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:44.908958 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:44.908958 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:44.908958 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:44.908958 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:44.908958 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:44.908958 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:44.908958 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:44.908958 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:44.908958 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 07:20:44.908958 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 07:20:44.908958 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 07:20:44.908958 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 07:20:44.909922 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:44.909922 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:44.909922 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 07:20:44.909922 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 07:20:44.909922 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 07:20:44.909922 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 07:20:44.909922 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:44.909922 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:44.909922 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:44.909922 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:44.909922 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:44.909922 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:44.909922 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:44.909922 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:44.909922 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:44.909922 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:44.909922 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:44.909922 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:44.910857 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:44.910857 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:44.910857 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 07:20:44.910857 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 07:20:44.910857 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 07:20:44.910857 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 07:20:44.910857 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 07:20:44.910857 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 07:20:44.910857 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 15 07:20:44.910857 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:20:44.910857 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 291/575] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 15 07:20:44 2022] GET /identity/v3/users/cinder => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:44.912130 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2b43bf-2878-44ee-812c-b614d7459aff None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:44.955920 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:45.012062 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea738358-99a6-487e-8b49-7f86f4a172db None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:45.019220 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2b43bf-2878-44ee-812c-b614d7459aff None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:45.047733 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff0557e-adfe-4086-9099-b09d25a6bae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:45.048368 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff0557e-adfe-4086-9099-b09d25a6bae9 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:45.048648 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff0557e-adfe-4086-9099-b09d25a6bae9 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:45.048926 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff0557e-adfe-4086-9099-b09d25a6bae9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:45.052859 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37335a0-905c-4315-a13e-6f2b91aa25a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:45.053478 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37335a0-905c-4315-a13e-6f2b91aa25a9 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:45.053798 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37335a0-905c-4315-a13e-6f2b91aa25a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:45.054074 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37335a0-905c-4315-a13e-6f2b91aa25a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:45.058661 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff0557e-adfe-4086-9099-b09d25a6bae9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:45.060350 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff0557e-adfe-4086-9099-b09d25a6bae9 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:45.064537 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37335a0-905c-4315-a13e-6f2b91aa25a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:45.066127 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37335a0-905c-4315-a13e-6f2b91aa25a9 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:45.075739 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-7ff0557e-adfe-4086-9099-b09d25a6bae9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:20:45.075739 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:45.075739 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:45.075739 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:45.075739 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:45.075739 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:45.075739 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:45.075739 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:45.075739 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:45.075739 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:45.075739 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:45.075739 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:45.075739 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 07:20:45.075739 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 07:20:45.075739 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 07:20:45.075739 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 07:20:45.076660 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:45.076660 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:45.076660 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:45.076660 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:45.076660 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:45.076660 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:45.076660 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:45.076660 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:45.076660 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:45.076660 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:45.076660 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:45.076660 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:45.076660 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:45.076660 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:45.076660 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:45.076660 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:45.076660 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 07:20:45.078632 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 07:20:45.078632 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:20:45.078632 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:20:45.078632 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 285/576] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:20:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:45.078632 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff0557e-adfe-4086-9099-b09d25a6bae9 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:45.080049 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-d37335a0-905c-4315-a13e-6f2b91aa25a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:20:45.080049 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:45.080049 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:45.080049 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:45.080049 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:45.080049 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:45.080049 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:45.080049 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:45.080049 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:45.080049 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:45.080049 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:45.080049 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:45.080049 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 07:20:45.080049 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 07:20:45.080049 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 07:20:45.080049 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 07:20:45.080988 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:45.080988 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:45.080988 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:45.080988 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:45.080988 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:45.080988 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:45.080988 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:45.080988 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:45.080988 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:45.080988 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:45.080988 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:45.080988 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:45.080988 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:45.080988 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:45.080988 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:45.080988 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:45.080988 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 07:20:45.083464 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 07:20:45.083464 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:20:45.083464 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:20:45.083464 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 292/577] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:20:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:45.083464 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37335a0-905c-4315-a13e-6f2b91aa25a9 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:45.126413 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:45.172877 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:45.181152 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff0557e-adfe-4086-9099-b09d25a6bae9 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:45.182031 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37335a0-905c-4315-a13e-6f2b91aa25a9 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:45.214877 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8cfbd6-bcb8-4e03-af3e-ee2c6d2954ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:45.215383 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7696c869-db99-4ba4-ab17-7d64412ea720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:45.215624 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8cfbd6-bcb8-4e03-af3e-ee2c6d2954ed None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:45.215949 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8cfbd6-bcb8-4e03-af3e-ee2c6d2954ed None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:45.216054 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7696c869-db99-4ba4-ab17-7d64412ea720 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:45.216258 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7696c869-db99-4ba4-ab17-7d64412ea720 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:45.216415 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8cfbd6-bcb8-4e03-af3e-ee2c6d2954ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:45.216536 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7696c869-db99-4ba4-ab17-7d64412ea720 None admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:45.217262 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7696c869-db99-4ba4-ab17-7d64412ea720 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:45.217570 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7696c869-db99-4ba4-ab17-7d64412ea720 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:45.218327 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8cfbd6-bcb8-4e03-af3e-ee2c6d2954ed None admin] RBAC: Adding query filter params (role.id=0e661b5a67664692b933e5da9fc2e488, scope.project.id=4e9987720930473da840579654311bc2, user.id=72b90d35087e4d7a80e9417082a1541f) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:45.218621 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8cfbd6-bcb8-4e03-af3e-ee2c6d2954ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:45.219721 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7696c869-db99-4ba4-ab17-7d64412ea720 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:45.220909 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8cfbd6-bcb8-4e03-af3e-ee2c6d2954ed None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:45.226388 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 286/578] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 15 07:20:45 2022] GET /identity/v3/role_assignments?user.id=72b90d35087e4d7a80e9417082a1541f&scope.project.id=4e9987720930473da840579654311bc2&role.id=0e661b5a67664692b933e5da9fc2e488 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:45.227172 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8cfbd6-bcb8-4e03-af3e-ee2c6d2954ed None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:45.238514 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 293/579] 10.222.0.42 () {56 vars in 1137 bytes} [Thu Sep 15 07:20:45 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:45.239409 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7696c869-db99-4ba4-ab17-7d64412ea720 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:45.351659 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7696c869-db99-4ba4-ab17-7d64412ea720 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:45.353441 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8cfbd6-bcb8-4e03-af3e-ee2c6d2954ed None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:45.363765 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:45.389226 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb19dbcf-4983-47a3-8a9b-ad52aa882d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:45.390397 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb19dbcf-4983-47a3-8a9b-ad52aa882d27 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:45.390733 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb19dbcf-4983-47a3-8a9b-ad52aa882d27 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:45.391007 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb19dbcf-4983-47a3-8a9b-ad52aa882d27 None admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:45.391451 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46980fa-c652-470e-85d4-9fdbab1f949e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:45.391646 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb19dbcf-4983-47a3-8a9b-ad52aa882d27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:45.391942 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb19dbcf-4983-47a3-8a9b-ad52aa882d27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:45.392097 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46980fa-c652-470e-85d4-9fdbab1f949e None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:45.392364 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46980fa-c652-470e-85d4-9fdbab1f949e None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:45.392637 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46980fa-c652-470e-85d4-9fdbab1f949e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:45.393331 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb19dbcf-4983-47a3-8a9b-ad52aa882d27 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:45.404369 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46980fa-c652-470e-85d4-9fdbab1f949e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:20:45.408187 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 294/580] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 15 07:20:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:45.409250 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb19dbcf-4983-47a3-8a9b-ad52aa882d27 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:45.410383 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-e46980fa-c652-470e-85d4-9fdbab1f949e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 07:20:45.410383 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:45.410383 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:45.410383 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:45.410383 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:45.410383 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:45.410383 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:45.410383 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:45.410383 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:45.410383 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:45.410383 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:45.410383 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:45.410383 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:20:45.410383 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:20:45.410383 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:20:45.410383 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:20:45.411362 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:20:45.411362 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:20:45.411362 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:20:45.411362 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:20:45.411362 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:45.411362 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:45.411362 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:45.411362 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:45.411362 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:45.411362 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:45.411362 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:45.411362 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:45.411362 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:45.411362 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:45.411362 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:45.411362 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:45.412182 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:45.412182 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:45.412182 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:45.412182 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:45.412182 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:20:45.412182 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:20:45.412182 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:20:45.412182 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:20:45.412182 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:20:45.412182 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:20:45.412182 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 07:20:45.412182 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:20:45.416016 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 287/581] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Sep 15 07:20:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:45.416835 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46980fa-c652-470e-85d4-9fdbab1f949e None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:45.429166 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:45.522442 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb19dbcf-4983-47a3-8a9b-ad52aa882d27 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:45.528236 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46980fa-c652-470e-85d4-9fdbab1f949e None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:45.557211 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4dae0e-75e1-4d75-bf3e-737055a91971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:45.557851 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4dae0e-75e1-4d75-bf3e-737055a91971 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:45.558129 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4dae0e-75e1-4d75-bf3e-737055a91971 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:45.558400 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4dae0e-75e1-4d75-bf3e-737055a91971 None admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:45.559301 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4dae0e-75e1-4d75-bf3e-737055a91971 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:45.559604 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4dae0e-75e1-4d75-bf3e-737055a91971 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:45.560962 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4dae0e-75e1-4d75-bf3e-737055a91971 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:45.566599 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4b0be7-c7e7-4a2c-93b8-f8fdbd9889b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:45.567240 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4b0be7-c7e7-4a2c-93b8-f8fdbd9889b3 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:45.567525 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4b0be7-c7e7-4a2c-93b8-f8fdbd9889b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:45.568287 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4b0be7-c7e7-4a2c-93b8-f8fdbd9889b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:45.576956 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 295/582] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 15 07:20:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:45.576956 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4dae0e-75e1-4d75-bf3e-737055a91971 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:45.583314 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4b0be7-c7e7-4a2c-93b8-f8fdbd9889b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:45.584851 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4b0be7-c7e7-4a2c-93b8-f8fdbd9889b3 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:45.601478 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-fd4b0be7-c7e7-4a2c-93b8-f8fdbd9889b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:20:45.601478 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:45.601478 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:45.601478 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:45.601478 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:45.601478 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:45.601478 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:45.601478 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:45.601478 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:45.601478 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:45.601478 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:45.601478 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:45.601478 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 07:20:45.601478 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 07:20:45.601478 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 07:20:45.601478 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 07:20:45.602519 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:45.602519 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:45.602519 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:45.602519 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:45.602519 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:45.602519 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:45.602519 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:45.602519 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:45.602519 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:45.602519 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:45.602519 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:45.602519 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:45.602519 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:45.602519 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:45.602519 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:45.602519 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:45.602519 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 07:20:45.603424 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 07:20:45.603424 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:20:45.603424 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:20:45.603424 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 288/583] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:20:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:45.603424 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4b0be7-c7e7-4a2c-93b8-f8fdbd9889b3 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:45.627419 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:45.653534 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:45.690123 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4dae0e-75e1-4d75-bf3e-737055a91971 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:45.723594 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4b0be7-c7e7-4a2c-93b8-f8fdbd9889b3 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:45.730991 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2237e5e6-53a9-434b-a4fe-fe236d7fa870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:45.731632 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2237e5e6-53a9-434b-a4fe-fe236d7fa870 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:45.732219 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2237e5e6-53a9-434b-a4fe-fe236d7fa870 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:45.732493 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2237e5e6-53a9-434b-a4fe-fe236d7fa870 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:45.742932 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2237e5e6-53a9-434b-a4fe-fe236d7fa870 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:45.744386 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2237e5e6-53a9-434b-a4fe-fe236d7fa870 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:45.756544 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-2237e5e6-53a9-434b-a4fe-fe236d7fa870 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 07:20:45.756544 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:45.756544 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:45.756544 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:45.756544 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:45.756544 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:45.756544 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:45.756544 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:45.756544 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:45.756544 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:45.756544 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:45.756544 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:45.756544 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 07:20:45.756544 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 07:20:45.756544 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 07:20:45.757688 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 07:20:45.757688 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:45.757688 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:45.757688 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:45.757688 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:45.757688 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:45.757688 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:45.757688 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:45.757688 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:45.757688 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:45.757688 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:45.757688 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:45.757688 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:45.757688 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:45.757688 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:45.757688 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:45.757688 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:45.758592 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 07:20:45.758592 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 07:20:45.758592 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 07:20:45.758592 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 07:20:45.758592 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 07:20:45.758592 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 07:20:45.758592 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 07:20:45.758592 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:20:45.758592 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 296/584] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 15 07:20:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:45.758592 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2237e5e6-53a9-434b-a4fe-fe236d7fa870 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:45.772201 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac69558e-0516-4d19-8681-8635f0d57ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:45.772817 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac69558e-0516-4d19-8681-8635f0d57ac4 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:45.773101 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac69558e-0516-4d19-8681-8635f0d57ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:45.773638 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac69558e-0516-4d19-8681-8635f0d57ac4 None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:45.774128 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac69558e-0516-4d19-8681-8635f0d57ac4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:45.774440 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac69558e-0516-4d19-8681-8635f0d57ac4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:45.775818 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac69558e-0516-4d19-8681-8635f0d57ac4 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:45.789424 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 289/585] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Sep 15 07:20:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:45.789424 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac69558e-0516-4d19-8681-8635f0d57ac4 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:45.871577 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2237e5e6-53a9-434b-a4fe-fe236d7fa870 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:45.880087 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:45.898320 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:45.899958 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac69558e-0516-4d19-8681-8635f0d57ac4 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:45.906650 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e88cf5c-218b-42f8-87ef-245ee1d6e9aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:45.907283 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e88cf5c-218b-42f8-87ef-245ee1d6e9aa None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:45.907565 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e88cf5c-218b-42f8-87ef-245ee1d6e9aa None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:45.907834 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e88cf5c-218b-42f8-87ef-245ee1d6e9aa None admin] PATH_INFO: `/v3/users/nova` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:45.916523 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e88cf5c-218b-42f8-87ef-245ee1d6e9aa None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:45.917998 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e88cf5c-218b-42f8-87ef-245ee1d6e9aa None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:45.929207 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-5e88cf5c-218b-42f8-87ef-245ee1d6e9aa None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 15 07:20:45.929207 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:45.929207 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:45.929207 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:45.929207 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:45.929207 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:45.929207 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:45.929207 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:45.929207 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:45.929207 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:45.929207 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:45.929207 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:45.929207 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 07:20:45.929207 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 07:20:45.929207 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 07:20:45.929207 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 07:20:45.930173 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:45.930173 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:45.930173 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 07:20:45.930173 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 07:20:45.930173 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 07:20:45.930173 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 07:20:45.930173 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:45.930173 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:45.930173 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:45.930173 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:45.930173 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:45.930173 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:45.930173 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:45.930173 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:45.930173 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:45.930173 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:45.930173 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:45.930173 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:45.931172 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:45.931172 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:45.931172 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 07:20:45.931172 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 07:20:45.931172 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 07:20:45.931172 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 07:20:45.931172 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 07:20:45.931172 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 07:20:45.931172 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 15 07:20:45.931172 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:20:45.931172 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 297/586] 10.222.0.42 () {56 vars in 1093 bytes} [Thu Sep 15 07:20:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:45.931172 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e88cf5c-218b-42f8-87ef-245ee1d6e9aa None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:45.954707 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa347641-6519-48ae-87e9-b01ac066a29d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:45.955637 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa347641-6519-48ae-87e9-b01ac066a29d None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:45.956062 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa347641-6519-48ae-87e9-b01ac066a29d None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:45.956502 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa347641-6519-48ae-87e9-b01ac066a29d None admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:45.957291 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa347641-6519-48ae-87e9-b01ac066a29d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:45.957826 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa347641-6519-48ae-87e9-b01ac066a29d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:45.959192 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa347641-6519-48ae-87e9-b01ac066a29d None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:45.976182 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 290/587] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 15 07:20:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:45.976182 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa347641-6519-48ae-87e9-b01ac066a29d None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:46.061478 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e88cf5c-218b-42f8-87ef-245ee1d6e9aa None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:46.092372 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa347641-6519-48ae-87e9-b01ac066a29d None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:46.099712 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2c8bba-b847-47fa-bbae-04032696d95d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:46.100317 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2c8bba-b847-47fa-bbae-04032696d95d None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:46.100602 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2c8bba-b847-47fa-bbae-04032696d95d None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:46.100876 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2c8bba-b847-47fa-bbae-04032696d95d None admin] PATH_INFO: `/v3/projects` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:46.101568 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2c8bba-b847-47fa-bbae-04032696d95d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:46.101900 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2c8bba-b847-47fa-bbae-04032696d95d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:46.103344 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2c8bba-b847-47fa-bbae-04032696d95d None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:46.116884 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 298/588] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 15 07:20:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:46.117785 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2c8bba-b847-47fa-bbae-04032696d95d None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:46.136233 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946559d5-8f70-4ea4-a1b7-fd3c4d8ad55c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:46.136961 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946559d5-8f70-4ea4-a1b7-fd3c4d8ad55c None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:46.137319 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946559d5-8f70-4ea4-a1b7-fd3c4d8ad55c None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:46.137701 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946559d5-8f70-4ea4-a1b7-fd3c4d8ad55c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:46.148675 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946559d5-8f70-4ea4-a1b7-fd3c4d8ad55c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:46.150166 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946559d5-8f70-4ea4-a1b7-fd3c4d8ad55c None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:46.151885 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:46.166276 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-946559d5-8f70-4ea4-a1b7-fd3c4d8ad55c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:20:46.166276 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:46.166276 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:46.166276 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:46.166276 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:46.166276 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:46.166276 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:46.166276 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:46.166276 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:46.166276 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:46.166276 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:46.166276 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:46.166276 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 07:20:46.166276 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 07:20:46.166276 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 07:20:46.166276 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 07:20:46.167490 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:46.167490 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:46.167490 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:46.167490 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:46.167490 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:46.167490 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:46.167490 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:46.167490 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:46.167490 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:46.167490 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:46.167490 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:46.167490 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:46.167490 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:46.167490 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:46.167490 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:46.167490 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:46.167490 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 07:20:46.173742 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 07:20:46.173742 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:20:46.173742 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:20:46.173742 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 291/589] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:20:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:46.173742 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946559d5-8f70-4ea4-a1b7-fd3c4d8ad55c None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:46.179493 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:46.240560 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2c8bba-b847-47fa-bbae-04032696d95d None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:46.278470 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26146d78-c266-4d42-9b15-5d5082b27dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:46.279165 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26146d78-c266-4d42-9b15-5d5082b27dc2 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:46.279523 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26146d78-c266-4d42-9b15-5d5082b27dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:46.279876 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26146d78-c266-4d42-9b15-5d5082b27dc2 None admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:46.280402 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946559d5-8f70-4ea4-a1b7-fd3c4d8ad55c None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:46.281006 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26146d78-c266-4d42-9b15-5d5082b27dc2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:46.281449 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26146d78-c266-4d42-9b15-5d5082b27dc2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:46.283236 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26146d78-c266-4d42-9b15-5d5082b27dc2 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:46.302269 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 299/590] 10.222.0.42 () {56 vars in 1133 bytes} [Thu Sep 15 07:20:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:46.303162 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26146d78-c266-4d42-9b15-5d5082b27dc2 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:46.323512 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205aefdf-d389-4f3f-a79f-19e32701b8ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:46.324189 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205aefdf-d389-4f3f-a79f-19e32701b8ff None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:46.324529 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205aefdf-d389-4f3f-a79f-19e32701b8ff None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:46.324807 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205aefdf-d389-4f3f-a79f-19e32701b8ff None admin] PATH_INFO: `/v3/projects/service` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:46.332304 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205aefdf-d389-4f3f-a79f-19e32701b8ff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:46.333771 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205aefdf-d389-4f3f-a79f-19e32701b8ff None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:46.345838 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-205aefdf-d389-4f3f-a79f-19e32701b8ff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 07:20:46.345838 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:46.345838 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:46.345838 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:46.345838 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:46.345838 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:46.345838 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:46.345838 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:46.345838 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:46.345838 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:46.345838 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:46.345838 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:46.345838 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 07:20:46.345838 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 07:20:46.345838 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 07:20:46.346868 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 07:20:46.346868 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:46.346868 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:46.346868 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:46.346868 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:46.346868 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:46.346868 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:46.346868 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:46.346868 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:46.346868 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:46.346868 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:46.346868 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:46.346868 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:46.346868 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:46.346868 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:46.346868 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:46.346868 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:46.348025 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 07:20:46.348025 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 07:20:46.348025 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 07:20:46.348025 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 07:20:46.348025 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 07:20:46.348025 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 07:20:46.348025 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 07:20:46.348025 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:20:46.351421 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 292/591] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 15 07:20:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:46.352241 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205aefdf-d389-4f3f-a79f-19e32701b8ff None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:46.411461 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:46.421299 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26146d78-c266-4d42-9b15-5d5082b27dc2 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:46.429693 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:46.459376 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71d7ef5-b376-4e0c-b5c7-8548b1e3f8d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:46.460120 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71d7ef5-b376-4e0c-b5c7-8548b1e3f8d2 None admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:46.460385 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205aefdf-d389-4f3f-a79f-19e32701b8ff None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:46.460506 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71d7ef5-b376-4e0c-b5c7-8548b1e3f8d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:46.460867 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71d7ef5-b376-4e0c-b5c7-8548b1e3f8d2 None admin] PATH_INFO: `/v3/projects/4e9987720930473da840579654311bc2/users/37438b032a1f46d3bdf7121ad0cffbc3/roles/0e661b5a67664692b933e5da9fc2e488` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:46.476331 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71d7ef5-b376-4e0c-b5c7-8548b1e3f8d2 None admin] RBAC: Authorizing `identity:create_grant(project_id=4e9987720930473da840579654311bc2, user_id=37438b032a1f46d3bdf7121ad0cffbc3, role_id=0e661b5a67664692b933e5da9fc2e488)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:46.477913 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71d7ef5-b376-4e0c-b5c7-8548b1e3f8d2 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:46.495981 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17216776-8adb-4750-9460-5134848617c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:46.496589 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17216776-8adb-4750-9460-5134848617c9 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:46.496870 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17216776-8adb-4750-9460-5134848617c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:46.497142 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17216776-8adb-4750-9460-5134848617c9 None admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:46.497808 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17216776-8adb-4750-9460-5134848617c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:46.498111 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17216776-8adb-4750-9460-5134848617c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:46.500153 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17216776-8adb-4750-9460-5134848617c9 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:46.511838 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 293/592] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 15 07:20:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:46.512629 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17216776-8adb-4750-9460-5134848617c9 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:46.543549 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 300/593] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Sep 15 07:20:46 2022] PUT /identity/v3/projects/4e9987720930473da840579654311bc2/users/37438b032a1f46d3bdf7121ad0cffbc3/roles/0e661b5a67664692b933e5da9fc2e488 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:20:46.554930 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71d7ef5-b376-4e0c-b5c7-8548b1e3f8d2 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:46.631552 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17216776-8adb-4750-9460-5134848617c9 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:46.650188 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:46.677110 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df17f183-00c4-426d-b470-7f9bfcd8b487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:46.677987 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df17f183-00c4-426d-b470-7f9bfcd8b487 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:46.678266 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df17f183-00c4-426d-b470-7f9bfcd8b487 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:46.678543 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df17f183-00c4-426d-b470-7f9bfcd8b487 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:46.694621 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df17f183-00c4-426d-b470-7f9bfcd8b487 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:46.696481 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df17f183-00c4-426d-b470-7f9bfcd8b487 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:46.716392 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-df17f183-00c4-426d-b470-7f9bfcd8b487 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:20:46.716392 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:46.716392 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:46.716392 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:46.716392 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:46.716392 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:46.716392 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:46.716392 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:46.716392 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:46.716392 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:46.716392 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:46.716392 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:46.716392 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 07:20:46.716392 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 07:20:46.716392 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 07:20:46.716392 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 07:20:46.717607 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:46.717607 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:46.717607 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:46.717607 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:46.717607 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:46.717607 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:46.717607 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:46.717607 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:46.717607 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:46.717607 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:46.717607 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:46.717607 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:46.717607 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:46.717607 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:46.717607 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:46.717607 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:46.717607 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 07:20:46.719281 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 07:20:46.719281 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:20:46.719281 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:20:46.722677 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71d7ef5-b376-4e0c-b5c7-8548b1e3f8d2 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:46.723427 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 294/594] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:20:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:46.724452 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df17f183-00c4-426d-b470-7f9bfcd8b487 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:46.770961 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3390e898-6a47-416e-bc7d-a713e55d2055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:46.770961 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3390e898-6a47-416e-bc7d-a713e55d2055 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:46.771429 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3390e898-6a47-416e-bc7d-a713e55d2055 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:46.771792 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3390e898-6a47-416e-bc7d-a713e55d2055 None admin] PATH_INFO: `/v3/projects` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:46.772584 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3390e898-6a47-416e-bc7d-a713e55d2055 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:46.772985 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3390e898-6a47-416e-bc7d-a713e55d2055 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:46.774690 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3390e898-6a47-416e-bc7d-a713e55d2055 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:46.789522 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 301/595] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 15 07:20:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:46.790619 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3390e898-6a47-416e-bc7d-a713e55d2055 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:46.815021 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:46.869437 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df17f183-00c4-426d-b470-7f9bfcd8b487 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:46.902292 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3390e898-6a47-416e-bc7d-a713e55d2055 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:46.919609 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfbc95f-41f8-4eba-a64d-23589c7519db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:46.920258 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfbc95f-41f8-4eba-a64d-23589c7519db None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:46.920543 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfbc95f-41f8-4eba-a64d-23589c7519db None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:46.920842 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfbc95f-41f8-4eba-a64d-23589c7519db None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:46.933204 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfbc95f-41f8-4eba-a64d-23589c7519db None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:46.935535 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfbc95f-41f8-4eba-a64d-23589c7519db None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:46.944289 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da76f0d-932e-4935-92f2-3741844144c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:46.945166 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da76f0d-932e-4935-92f2-3741844144c1 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:46.945628 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da76f0d-932e-4935-92f2-3741844144c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:46.946134 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da76f0d-932e-4935-92f2-3741844144c1 None admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:46.946965 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da76f0d-932e-4935-92f2-3741844144c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:46.947397 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da76f0d-932e-4935-92f2-3741844144c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:46.949040 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da76f0d-932e-4935-92f2-3741844144c1 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:46.954622 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-1cfbc95f-41f8-4eba-a64d-23589c7519db None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 15 07:20:46.954622 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:46.954622 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:46.954622 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:46.954622 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:46.954622 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:46.954622 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:46.954622 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:46.954622 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:46.954622 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:46.954622 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:46.954622 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:46.954622 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 07:20:46.954622 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 07:20:46.954622 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 07:20:46.954622 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 07:20:46.955703 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:46.955703 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:46.955703 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 07:20:46.955703 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 07:20:46.955703 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 07:20:46.955703 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 07:20:46.955703 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:46.955703 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:46.955703 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:46.955703 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:46.955703 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:46.955703 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:46.955703 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:46.955703 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:46.955703 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:46.955703 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:46.955703 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:46.955703 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:46.956868 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:46.956868 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:46.956868 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 07:20:46.956868 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 07:20:46.956868 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 07:20:46.956868 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 07:20:46.956868 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 07:20:46.956868 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 07:20:46.956868 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 15 07:20:46.956868 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:20:46.956868 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 295/596] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Sep 15 07:20:46 2022] GET /identity/v3/users/neutron => generated 85 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:46.957549 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfbc95f-41f8-4eba-a64d-23589c7519db None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:46.966101 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 302/597] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 15 07:20:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:46.967221 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da76f0d-932e-4935-92f2-3741844144c1 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:46.998557 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:47.054704 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:47.079442 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da76f0d-932e-4935-92f2-3741844144c1 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:47.096716 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfbc95f-41f8-4eba-a64d-23589c7519db None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:47.119558 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8353797-398e-47d0-9f85-163185b59161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:47.120400 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8353797-398e-47d0-9f85-163185b59161 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:47.120823 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8353797-398e-47d0-9f85-163185b59161 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:47.121235 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8353797-398e-47d0-9f85-163185b59161 None admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:47.122136 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8353797-398e-47d0-9f85-163185b59161 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:47.122607 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8353797-398e-47d0-9f85-163185b59161 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:47.124307 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8353797-398e-47d0-9f85-163185b59161 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:47.138729 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89038bd-e323-420f-b23d-51f8015a418b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:47.140489 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89038bd-e323-420f-b23d-51f8015a418b None admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:47.140774 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89038bd-e323-420f-b23d-51f8015a418b None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:47.141057 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89038bd-e323-420f-b23d-51f8015a418b None admin] PATH_INFO: `/v3/projects/4e9987720930473da840579654311bc2/users/3ecf6642b7b84d3c9756bb41ab3349d3/roles/0e661b5a67664692b933e5da9fc2e488` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:47.144477 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 303/598] 10.222.0.42 () {56 vars in 1139 bytes} [Thu Sep 15 07:20:46 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:47.145789 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8353797-398e-47d0-9f85-163185b59161 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:47.159639 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89038bd-e323-420f-b23d-51f8015a418b None admin] RBAC: Authorizing `identity:create_grant(project_id=4e9987720930473da840579654311bc2, user_id=3ecf6642b7b84d3c9756bb41ab3349d3, role_id=0e661b5a67664692b933e5da9fc2e488)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:47.161095 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89038bd-e323-420f-b23d-51f8015a418b None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:47.290585 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8353797-398e-47d0-9f85-163185b59161 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:47.293796 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 296/599] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Sep 15 07:20:46 2022] PUT /identity/v3/projects/4e9987720930473da840579654311bc2/users/3ecf6642b7b84d3c9756bb41ab3349d3/roles/0e661b5a67664692b933e5da9fc2e488 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:20:47.295427 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89038bd-e323-420f-b23d-51f8015a418b None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:47.342626 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c27f02f-deda-4b4b-a4ae-fda83918fb73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:47.344930 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c27f02f-deda-4b4b-a4ae-fda83918fb73 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:47.345460 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c27f02f-deda-4b4b-a4ae-fda83918fb73 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:47.345973 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c27f02f-deda-4b4b-a4ae-fda83918fb73 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:47.348758 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:47.356382 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c27f02f-deda-4b4b-a4ae-fda83918fb73 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:47.358485 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c27f02f-deda-4b4b-a4ae-fda83918fb73 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:47.376351 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-4c27f02f-deda-4b4b-a4ae-fda83918fb73 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 07:20:47.376351 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:47.376351 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:47.376351 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:47.376351 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:47.376351 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:47.376351 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:47.376351 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:47.376351 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:47.376351 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:47.376351 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:47.376351 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:47.376351 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 07:20:47.376351 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 07:20:47.376351 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 07:20:47.377421 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 07:20:47.377421 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:47.377421 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:47.377421 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:47.377421 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:47.377421 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:47.377421 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:47.377421 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:47.377421 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:47.377421 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:47.377421 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:47.377421 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:47.377421 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:47.377421 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:47.377421 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:47.377421 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:47.377421 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:47.378336 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 07:20:47.378336 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 07:20:47.378336 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 07:20:47.378336 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 07:20:47.378336 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 07:20:47.378336 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 07:20:47.378336 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 07:20:47.378336 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:20:47.378336 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 304/600] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 15 07:20:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:47.387316 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c27f02f-deda-4b4b-a4ae-fda83918fb73 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:47.455323 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:47.479766 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89038bd-e323-420f-b23d-51f8015a418b None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:47.519374 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c27f02f-deda-4b4b-a4ae-fda83918fb73 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:47.522369 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43315bc9-e2c9-4808-818e-283dc7b28762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:47.523293 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43315bc9-e2c9-4808-818e-283dc7b28762 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:47.523731 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43315bc9-e2c9-4808-818e-283dc7b28762 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:47.524154 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43315bc9-e2c9-4808-818e-283dc7b28762 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:47.538410 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43315bc9-e2c9-4808-818e-283dc7b28762 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:47.539886 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43315bc9-e2c9-4808-818e-283dc7b28762 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:47.558593 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-43315bc9-e2c9-4808-818e-283dc7b28762 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:20:47.558593 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:47.558593 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:47.558593 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:47.558593 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:47.558593 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:47.558593 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:47.558593 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:47.558593 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:47.558593 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:47.558593 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:47.558593 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:47.558593 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 07:20:47.558593 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 07:20:47.558593 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 07:20:47.558593 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 07:20:47.559863 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:47.559863 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:47.559863 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:47.559863 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:47.559863 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:47.559863 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:47.559863 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:47.559863 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:47.559863 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:47.559863 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:47.559863 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:47.559863 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:47.559863 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:47.559863 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:47.559863 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:47.559863 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:47.559863 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 07:20:47.560768 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 07:20:47.560768 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:20:47.560768 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:20:47.560768 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 297/601] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:20:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:47.561533 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e69889-ca60-41ce-a6ed-03abf8facc28 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:47.561533 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e69889-ca60-41ce-a6ed-03abf8facc28 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:47.561860 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e69889-ca60-41ce-a6ed-03abf8facc28 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:47.563821 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 298/602] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:20:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:20:47.566832 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf42f57e-4c97-4b04-911b-048caf05d1de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:47.567189 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf42f57e-4c97-4b04-911b-048caf05d1de None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:47.567449 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf42f57e-4c97-4b04-911b-048caf05d1de None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:47.567730 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf42f57e-4c97-4b04-911b-048caf05d1de None admin] PATH_INFO: `/v3/projects` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:47.568422 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0df926-6a4e-4d83-95b2-7cedc8c58190 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:47.568520 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf42f57e-4c97-4b04-911b-048caf05d1de None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:47.568711 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0df926-6a4e-4d83-95b2-7cedc8c58190 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:47.568805 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf42f57e-4c97-4b04-911b-048caf05d1de None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:47.568988 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0df926-6a4e-4d83-95b2-7cedc8c58190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:47.570224 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf42f57e-4c97-4b04-911b-048caf05d1de None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:47.589615 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 305/603] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 15 07:20:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:47.589615 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf42f57e-4c97-4b04-911b-048caf05d1de None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:47.597776 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-5b0df926-6a4e-4d83-95b2-7cedc8c58190 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:20:47.599010 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-5b0df926-6a4e-4d83-95b2-7cedc8c58190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:20:47.653502 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:47.678875 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 299/604] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:20:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 07:20:47.679771 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0df926-6a4e-4d83-95b2-7cedc8c58190 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:47.737643 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf42f57e-4c97-4b04-911b-048caf05d1de None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:47.747808 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:47.794363 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0df926-6a4e-4d83-95b2-7cedc8c58190 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:47.828504 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a528d036-abc3-4ebd-9ea0-d7bdd2ec7437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:47.829301 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a528d036-abc3-4ebd-9ea0-d7bdd2ec7437 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:47.829604 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a528d036-abc3-4ebd-9ea0-d7bdd2ec7437 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:47.829898 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a528d036-abc3-4ebd-9ea0-d7bdd2ec7437 None admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:47.830545 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a528d036-abc3-4ebd-9ea0-d7bdd2ec7437 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:47.830953 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a528d036-abc3-4ebd-9ea0-d7bdd2ec7437 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:47.831370 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61c835c-cbbd-4dce-b59d-ac5d1b165a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:47.832090 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61c835c-cbbd-4dce-b59d-ac5d1b165a1a None admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:47.832328 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a528d036-abc3-4ebd-9ea0-d7bdd2ec7437 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:47.832430 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61c835c-cbbd-4dce-b59d-ac5d1b165a1a None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:47.832645 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61c835c-cbbd-4dce-b59d-ac5d1b165a1a None admin] PATH_INFO: `/v3/projects/4e9987720930473da840579654311bc2/users/a080c7ab9b684abca6828067f2572c3b/roles/0e661b5a67664692b933e5da9fc2e488` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:47.858453 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 306/605] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 15 07:20:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:47.862450 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61c835c-cbbd-4dce-b59d-ac5d1b165a1a None admin] RBAC: Authorizing `identity:create_grant(project_id=4e9987720930473da840579654311bc2, user_id=a080c7ab9b684abca6828067f2572c3b, role_id=0e661b5a67664692b933e5da9fc2e488)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:47.864005 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61c835c-cbbd-4dce-b59d-ac5d1b165a1a None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:47.867204 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a528d036-abc3-4ebd-9ea0-d7bdd2ec7437 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:47.984042 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 300/606] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Sep 15 07:20:47 2022] PUT /identity/v3/projects/4e9987720930473da840579654311bc2/users/a080c7ab9b684abca6828067f2572c3b/roles/0e661b5a67664692b933e5da9fc2e488 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:20:47.985597 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a528d036-abc3-4ebd-9ea0-d7bdd2ec7437 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:48.012032 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:48.036734 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f244bf25-0c3f-4f9d-a55b-f4ead326c75c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:48.037385 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f244bf25-0c3f-4f9d-a55b-f4ead326c75c None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:48.037746 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f244bf25-0c3f-4f9d-a55b-f4ead326c75c None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:48.038025 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f244bf25-0c3f-4f9d-a55b-f4ead326c75c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:48.046195 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f244bf25-0c3f-4f9d-a55b-f4ead326c75c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:48.047754 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f244bf25-0c3f-4f9d-a55b-f4ead326c75c None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:48.061310 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-f244bf25-0c3f-4f9d-a55b-f4ead326c75c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 07:20:48.061310 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:48.061310 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:48.061310 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:48.061310 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:48.061310 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:48.061310 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:48.061310 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:48.061310 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:48.061310 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:48.061310 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:48.061310 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:48.061310 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 07:20:48.061310 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 07:20:48.061310 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 07:20:48.062863 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 07:20:48.062863 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:48.062863 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:48.062863 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:48.062863 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:48.062863 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:48.062863 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:48.062863 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:48.062863 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:48.062863 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:48.062863 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:48.062863 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:48.062863 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:48.062863 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:48.062863 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:48.062863 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:48.062863 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:48.063714 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 07:20:48.063714 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 07:20:48.063714 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 07:20:48.063714 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 07:20:48.063714 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 07:20:48.063714 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 07:20:48.063714 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 07:20:48.063714 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:20:48.063714 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 307/607] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 15 07:20:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:48.070431 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61c835c-cbbd-4dce-b59d-ac5d1b165a1a None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:48.120524 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f244bf25-0c3f-4f9d-a55b-f4ead326c75c None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:48.220047 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61c835c-cbbd-4dce-b59d-ac5d1b165a1a None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:48.230241 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:48.257404 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f244bf25-0c3f-4f9d-a55b-f4ead326c75c None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:48.257639 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384738fc-1d2d-41c0-a74b-d3864f33ea99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:48.258138 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384738fc-1d2d-41c0-a74b-d3864f33ea99 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:48.258415 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384738fc-1d2d-41c0-a74b-d3864f33ea99 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:48.258688 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384738fc-1d2d-41c0-a74b-d3864f33ea99 None admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:48.259383 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384738fc-1d2d-41c0-a74b-d3864f33ea99 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:48.259687 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384738fc-1d2d-41c0-a74b-d3864f33ea99 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:48.261107 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384738fc-1d2d-41c0-a74b-d3864f33ea99 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:48.273245 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 301/608] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 15 07:20:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:20:48.278987 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384738fc-1d2d-41c0-a74b-d3864f33ea99 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:48.302570 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41724ce-13db-4b3e-a9c7-0a7a4664be67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:48.304000 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41724ce-13db-4b3e-a9c7-0a7a4664be67 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:48.304494 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41724ce-13db-4b3e-a9c7-0a7a4664be67 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:48.305075 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41724ce-13db-4b3e-a9c7-0a7a4664be67 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:48.321152 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41724ce-13db-4b3e-a9c7-0a7a4664be67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:20:48.329327 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-f41724ce-13db-4b3e-a9c7-0a7a4664be67 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 07:20:48.329327 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:48.329327 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:48.329327 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:48.329327 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:48.329327 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:48.329327 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:48.329327 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:48.329327 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:48.329327 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:48.329327 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:48.329327 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:48.329327 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:20:48.329327 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:20:48.329327 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:20:48.329327 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:20:48.330330 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:20:48.330330 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:20:48.330330 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:20:48.330330 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:20:48.330330 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:48.330330 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:48.330330 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:48.330330 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:48.330330 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:48.330330 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:48.330330 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:48.330330 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:48.330330 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:48.330330 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:48.330330 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:48.330330 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:48.331168 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:48.331168 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:48.331168 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:48.331168 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:48.331168 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:20:48.331168 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:20:48.331168 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:20:48.331168 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:20:48.331168 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:20:48.331168 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:20:48.331168 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 07:20:48.331168 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:20:48.337557 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 308/609] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Sep 15 07:20:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 07:20:48.340242 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41724ce-13db-4b3e-a9c7-0a7a4664be67 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:48.384575 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:48.462107 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384738fc-1d2d-41c0-a74b-d3864f33ea99 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:48.481816 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41724ce-13db-4b3e-a9c7-0a7a4664be67 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:48.533402 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d98e67-2ba5-4661-906c-cac63989657d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:48.534528 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d98e67-2ba5-4661-906c-cac63989657d None admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:48.535714 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d98e67-2ba5-4661-906c-cac63989657d None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:48.536816 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d98e67-2ba5-4661-906c-cac63989657d None admin] PATH_INFO: `/v3/projects/4e9987720930473da840579654311bc2/users/5315996432f44bffa3460b2744c10f0a/roles/0e661b5a67664692b933e5da9fc2e488` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:48.553055 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c12c55-6cd9-4089-aef3-6e6a837e75cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:48.554475 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c12c55-6cd9-4089-aef3-6e6a837e75cc None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:48.554988 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c12c55-6cd9-4089-aef3-6e6a837e75cc None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:48.555478 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c12c55-6cd9-4089-aef3-6e6a837e75cc None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:48.556395 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c12c55-6cd9-4089-aef3-6e6a837e75cc None admin] RBAC: Adding query filter params (name=service) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:48.556915 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c12c55-6cd9-4089-aef3-6e6a837e75cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:48.562643 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c12c55-6cd9-4089-aef3-6e6a837e75cc None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:48.572286 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d98e67-2ba5-4661-906c-cac63989657d None admin] RBAC: Authorizing `identity:create_grant(project_id=4e9987720930473da840579654311bc2, user_id=5315996432f44bffa3460b2744c10f0a, role_id=0e661b5a67664692b933e5da9fc2e488)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:48.574041 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d98e67-2ba5-4661-906c-cac63989657d None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:48.584461 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 309/610] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Sep 15 07:20:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:48.588569 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c12c55-6cd9-4089-aef3-6e6a837e75cc None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:48.694799 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:48.700424 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c12c55-6cd9-4089-aef3-6e6a837e75cc None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:48.734643 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a05004-9173-4fa7-b71a-3d84a300bacd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:48.735277 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a05004-9173-4fa7-b71a-3d84a300bacd None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:48.735560 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a05004-9173-4fa7-b71a-3d84a300bacd None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:48.735839 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a05004-9173-4fa7-b71a-3d84a300bacd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:48.743559 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 302/611] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Sep 15 07:20:48 2022] PUT /identity/v3/projects/4e9987720930473da840579654311bc2/users/5315996432f44bffa3460b2744c10f0a/roles/0e661b5a67664692b933e5da9fc2e488 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 07:20:48.753686 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a05004-9173-4fa7-b71a-3d84a300bacd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:48.755269 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a05004-9173-4fa7-b71a-3d84a300bacd None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:48.771104 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-a4a05004-9173-4fa7-b71a-3d84a300bacd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:20:48.771104 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:48.771104 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:48.771104 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:48.771104 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:48.771104 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:48.771104 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:48.771104 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:48.771104 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:48.771104 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:48.771104 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:48.771104 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:48.771104 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 07:20:48.771104 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 07:20:48.771104 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 07:20:48.771104 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 07:20:48.772678 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:48.772678 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:48.772678 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:48.772678 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:48.772678 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:48.772678 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:48.772678 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:48.772678 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:48.772678 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:48.772678 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:48.772678 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:48.772678 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:48.772678 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:48.772678 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:48.772678 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:48.772678 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:48.772678 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 07:20:48.773811 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 07:20:48.773811 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:20:48.773811 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:20:48.773811 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 310/612] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:20:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 07:20:48.778979 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d98e67-2ba5-4661-906c-cac63989657d None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:48.806809 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:48.929920 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d98e67-2ba5-4661-906c-cac63989657d None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:48.972140 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df3ae29-562f-4484-9f71-c59f74ef84ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:48.972685 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df3ae29-562f-4484-9f71-c59f74ef84ad None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:48.972971 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df3ae29-562f-4484-9f71-c59f74ef84ad None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:48.973245 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df3ae29-562f-4484-9f71-c59f74ef84ad None admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:48.973937 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df3ae29-562f-4484-9f71-c59f74ef84ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:48.974245 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df3ae29-562f-4484-9f71-c59f74ef84ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:48.975627 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df3ae29-562f-4484-9f71-c59f74ef84ad None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:48.988324 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 303/613] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 15 07:20:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:48.994690 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a05004-9173-4fa7-b71a-3d84a300bacd None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:49.003212 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f500c69-e02c-4135-b991-be546f44f82a None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:49.003513 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f500c69-e02c-4135-b991-be546f44f82a None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:49.003783 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f500c69-e02c-4135-b991-be546f44f82a None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:49.005381 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 304/614] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:20:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 07:20:49.012352 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9e284e-fb39-4016-a335-05957eaf5aae None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:49.012646 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9e284e-fb39-4016-a335-05957eaf5aae None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:49.012929 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9e284e-fb39-4016-a335-05957eaf5aae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:49.041060 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-1a9e284e-fb39-4016-a335-05957eaf5aae None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:20:49.042564 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-1a9e284e-fb39-4016-a335-05957eaf5aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:20:49.152544 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 305/615] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:20:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 07:20:49.161391 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a05004-9173-4fa7-b71a-3d84a300bacd None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:49.200480 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26595568-7df4-4042-87d0-4e5d01c50e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:49.201113 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26595568-7df4-4042-87d0-4e5d01c50e85 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:49.201397 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26595568-7df4-4042-87d0-4e5d01c50e85 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:49.201708 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26595568-7df4-4042-87d0-4e5d01c50e85 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:49.210996 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26595568-7df4-4042-87d0-4e5d01c50e85 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:49.212527 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26595568-7df4-4042-87d0-4e5d01c50e85 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:49.225456 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-26595568-7df4-4042-87d0-4e5d01c50e85 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 15 07:20:49.225456 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:49.225456 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:49.225456 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:49.225456 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:49.225456 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:49.225456 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:49.225456 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:49.225456 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:49.225456 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:49.225456 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:49.225456 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:49.225456 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 07:20:49.225456 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 07:20:49.225456 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 07:20:49.225456 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 07:20:49.226438 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:49.226438 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:49.226438 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 07:20:49.226438 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 07:20:49.226438 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 07:20:49.226438 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 07:20:49.226438 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:49.226438 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:49.226438 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:49.226438 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:49.226438 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:49.226438 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:49.226438 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:49.226438 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:49.226438 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:49.226438 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:49.226438 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:49.226438 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:49.227382 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:49.227382 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:49.227382 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 07:20:49.227382 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 07:20:49.227382 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 07:20:49.227382 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 07:20:49.227382 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 07:20:49.227382 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 07:20:49.227382 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 15 07:20:49.227382 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:20:49.231308 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 311/616] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 15 07:20:48 2022] GET /identity/v3/users/swift => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:49.232661 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9e284e-fb39-4016-a335-05957eaf5aae None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:49.257471 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:49.400418 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9e284e-fb39-4016-a335-05957eaf5aae None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:49.452655 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6ee305-3879-47a6-a53a-1280a85809c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:49.453666 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6ee305-3879-47a6-a53a-1280a85809c9 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:49.454171 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6ee305-3879-47a6-a53a-1280a85809c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:49.454658 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6ee305-3879-47a6-a53a-1280a85809c9 None admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:49.455612 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6ee305-3879-47a6-a53a-1280a85809c9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:49.456147 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6ee305-3879-47a6-a53a-1280a85809c9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:49.457783 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6ee305-3879-47a6-a53a-1280a85809c9 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:49.484555 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 306/617] 10.222.0.42 () {56 vars in 1135 bytes} [Thu Sep 15 07:20:49 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:49.492389 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26595568-7df4-4042-87d0-4e5d01c50e85 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:49.504762 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:49.509476 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6ee305-3879-47a6-a53a-1280a85809c9 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:49.584636 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:49.643727 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26595568-7df4-4042-87d0-4e5d01c50e85 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:49.653305 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6ee305-3879-47a6-a53a-1280a85809c9 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:49.694376 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf262f39-6b2b-41c0-b496-4ecbe386b3a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:49.695486 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf262f39-6b2b-41c0-b496-4ecbe386b3a4 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:49.695981 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf262f39-6b2b-41c0-b496-4ecbe386b3a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:49.696456 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf262f39-6b2b-41c0-b496-4ecbe386b3a4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:49.709311 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf262f39-6b2b-41c0-b496-4ecbe386b3a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:20:49.712995 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59932ba6-c906-4f89-8a6b-3b480fc50d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:49.713675 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59932ba6-c906-4f89-8a6b-3b480fc50d19 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:49.713957 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59932ba6-c906-4f89-8a6b-3b480fc50d19 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:49.714229 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59932ba6-c906-4f89-8a6b-3b480fc50d19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:49.716805 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-cf262f39-6b2b-41c0-b496-4ecbe386b3a4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 07:20:49.716805 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:49.716805 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:49.716805 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:49.716805 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:49.716805 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:49.716805 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:49.716805 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:49.716805 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:49.716805 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:49.716805 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:49.716805 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:49.716805 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:20:49.716805 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:20:49.716805 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:20:49.716805 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:20:49.717914 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:20:49.717914 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:20:49.717914 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:20:49.717914 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:20:49.717914 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:49.717914 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:49.717914 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:49.717914 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:49.717914 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:49.717914 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:49.717914 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:49.717914 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:49.717914 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:49.717914 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:49.717914 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:49.717914 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:49.719426 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:49.719426 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:49.719426 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:49.719426 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:49.719426 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:20:49.719426 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:20:49.719426 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:20:49.719426 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:20:49.719426 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:20:49.719426 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:20:49.719426 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 07:20:49.719426 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:20:49.722199 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 307/618] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Sep 15 07:20:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:49.734779 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf262f39-6b2b-41c0-b496-4ecbe386b3a4 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:49.734942 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59932ba6-c906-4f89-8a6b-3b480fc50d19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:49.736224 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59932ba6-c906-4f89-8a6b-3b480fc50d19 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:49.771083 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-59932ba6-c906-4f89-8a6b-3b480fc50d19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:20:49.771083 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:49.771083 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:49.771083 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:49.771083 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:49.771083 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:49.771083 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:49.771083 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:49.771083 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:49.771083 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:49.771083 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:49.771083 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:49.771083 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 07:20:49.771083 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 07:20:49.771083 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 07:20:49.771083 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 07:20:49.772115 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:49.772115 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:49.772115 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:49.772115 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:49.772115 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:49.772115 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:49.772115 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:49.772115 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:49.772115 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:49.772115 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:49.772115 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:49.772115 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:49.772115 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:49.772115 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:49.772115 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:49.772115 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:49.772115 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 07:20:49.773106 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 07:20:49.773106 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:20:49.773106 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:20:49.773106 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 312/619] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:20:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:49.781406 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59932ba6-c906-4f89-8a6b-3b480fc50d19 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:49.855700 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf262f39-6b2b-41c0-b496-4ecbe386b3a4 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:49.888003 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:49.902187 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6ae4f9-43f8-47ab-89f6-1b68fca3d2c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:49.903761 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ae4f9-43f8-47ab-89f6-1b68fca3d2c2 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:49.903761 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ae4f9-43f8-47ab-89f6-1b68fca3d2c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:49.905107 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:49.906156 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ae4f9-43f8-47ab-89f6-1b68fca3d2c2 None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:49.908067 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6ae4f9-43f8-47ab-89f6-1b68fca3d2c2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:49.909893 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6ae4f9-43f8-47ab-89f6-1b68fca3d2c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:49.913548 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6ae4f9-43f8-47ab-89f6-1b68fca3d2c2 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:49.932204 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 308/620] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Sep 15 07:20:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:49.943540 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6ae4f9-43f8-47ab-89f6-1b68fca3d2c2 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:49.995771 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59932ba6-c906-4f89-8a6b-3b480fc50d19 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:50.042527 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c2b340-8f2f-4144-a9c3-b0788ce33f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:50.043719 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c2b340-8f2f-4144-a9c3-b0788ce33f43 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:50.044630 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c2b340-8f2f-4144-a9c3-b0788ce33f43 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:50.045398 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c2b340-8f2f-4144-a9c3-b0788ce33f43 None admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:50.046875 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c2b340-8f2f-4144-a9c3-b0788ce33f43 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:50.047410 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c2b340-8f2f-4144-a9c3-b0788ce33f43 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:50.049289 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c2b340-8f2f-4144-a9c3-b0788ce33f43 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:50.067115 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 313/621] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 15 07:20:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:50.069542 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6963bd-0e6e-47f0-96ce-385df61fa42f None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:50.070074 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6963bd-0e6e-47f0-96ce-385df61fa42f None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:50.070791 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6963bd-0e6e-47f0-96ce-385df61fa42f None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:50.073387 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 314/622] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:20:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:20:50.074355 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6963bd-0e6e-47f0-96ce-385df61fa42f None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:50.095574 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6ae4f9-43f8-47ab-89f6-1b68fca3d2c2 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:50.138962 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8aabd5-cf06-4831-861d-bf773d2b7069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:50.139594 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8aabd5-cf06-4831-861d-bf773d2b7069 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:50.140165 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8aabd5-cf06-4831-861d-bf773d2b7069 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:50.140633 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8aabd5-cf06-4831-861d-bf773d2b7069 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:50.152437 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8aabd5-cf06-4831-861d-bf773d2b7069 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:50.154680 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8aabd5-cf06-4831-861d-bf773d2b7069 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:50.170646 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-ec8aabd5-cf06-4831-861d-bf773d2b7069 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:20:50.170646 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:50.170646 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:50.170646 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:50.170646 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:50.170646 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:50.170646 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:50.170646 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:50.170646 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:50.170646 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:50.170646 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:50.170646 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:50.170646 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 07:20:50.170646 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 07:20:50.170646 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 07:20:50.170646 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 07:20:50.172403 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:50.172403 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:50.172403 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:50.172403 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:50.172403 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:50.172403 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:50.172403 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:50.172403 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:50.172403 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:50.172403 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:50.172403 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:50.172403 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:50.172403 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:50.172403 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:50.172403 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:50.172403 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:50.172403 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 07:20:50.173283 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 07:20:50.173283 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:20:50.173283 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:20:50.173283 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 309/623] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:20:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:50.176945 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdf1738-b2e6-4cd4-ac9d-83348c979649 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:50.176945 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdf1738-b2e6-4cd4-ac9d-83348c979649 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:50.176945 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdf1738-b2e6-4cd4-ac9d-83348c979649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:50.196486 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-6cdf1738-b2e6-4cd4-ac9d-83348c979649 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:20:50.198149 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-6cdf1738-b2e6-4cd4-ac9d-83348c979649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:20:50.207170 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6963bd-0e6e-47f0-96ce-385df61fa42f None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:50.241072 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:50.243447 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:50.264066 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805d3a52-2870-4535-9a55-224bb798e6b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:50.264066 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805d3a52-2870-4535-9a55-224bb798e6b2 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:50.264066 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805d3a52-2870-4535-9a55-224bb798e6b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:50.264066 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805d3a52-2870-4535-9a55-224bb798e6b2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:50.269893 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805d3a52-2870-4535-9a55-224bb798e6b2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:50.271370 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805d3a52-2870-4535-9a55-224bb798e6b2 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:50.293640 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 310/624] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:20:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 07:20:50.293640 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdf1738-b2e6-4cd4-ac9d-83348c979649 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:50.296444 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-805d3a52-2870-4535-9a55-224bb798e6b2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 07:20:50.296444 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:50.296444 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:50.296444 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:50.296444 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:50.296444 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:50.296444 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:50.296444 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:50.296444 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:50.296444 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:50.296444 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:50.296444 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:50.296444 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 07:20:50.296444 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 07:20:50.296444 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 07:20:50.297706 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 07:20:50.297706 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:50.297706 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:50.297706 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:50.297706 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:50.297706 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:50.297706 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:50.297706 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:50.297706 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:50.297706 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:50.297706 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:50.297706 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:50.297706 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:50.297706 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:50.297706 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:50.297706 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:50.297706 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:50.302241 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 07:20:50.302241 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 07:20:50.302241 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 07:20:50.302241 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 07:20:50.302241 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 07:20:50.302241 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 07:20:50.302241 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 07:20:50.302241 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:20:50.307697 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 315/625] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 15 07:20:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:50.312929 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805d3a52-2870-4535-9a55-224bb798e6b2 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:50.490331 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdf1738-b2e6-4cd4-ac9d-83348c979649 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:50.501291 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805d3a52-2870-4535-9a55-224bb798e6b2 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:50.540531 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8bf771-418e-4504-b17e-430348c3e72a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:50.541483 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8bf771-418e-4504-b17e-430348c3e72a None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:50.542100 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8bf771-418e-4504-b17e-430348c3e72a None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:50.542512 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8bf771-418e-4504-b17e-430348c3e72a None admin] PATH_INFO: `/v3/projects` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:50.543346 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8bf771-418e-4504-b17e-430348c3e72a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:50.543812 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8bf771-418e-4504-b17e-430348c3e72a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:50.545602 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8bf771-418e-4504-b17e-430348c3e72a None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:50.564934 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 316/626] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 15 07:20:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:20:50.567222 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad69c457-ee02-4c58-a76a-6db38b13ec55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:50.568015 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad69c457-ee02-4c58-a76a-6db38b13ec55 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:50.568451 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad69c457-ee02-4c58-a76a-6db38b13ec55 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:50.568873 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad69c457-ee02-4c58-a76a-6db38b13ec55 None admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:50.569779 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad69c457-ee02-4c58-a76a-6db38b13ec55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:50.570233 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad69c457-ee02-4c58-a76a-6db38b13ec55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:50.571742 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad69c457-ee02-4c58-a76a-6db38b13ec55 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:50.572741 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8bf771-418e-4504-b17e-430348c3e72a None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:50.605262 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 311/627] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 15 07:20:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:50.611586 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad69c457-ee02-4c58-a76a-6db38b13ec55 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:50.626622 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:50.673226 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:50.690909 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8bf771-418e-4504-b17e-430348c3e72a None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:50.727908 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03bfb88-260d-460b-b484-a6b6926152f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:50.728716 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03bfb88-260d-460b-b484-a6b6926152f2 None admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:50.728834 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03bfb88-260d-460b-b484-a6b6926152f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:50.729111 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03bfb88-260d-460b-b484-a6b6926152f2 None admin] PATH_INFO: `/v3/projects/4e9987720930473da840579654311bc2/users/72b90d35087e4d7a80e9417082a1541f/roles/0e661b5a67664692b933e5da9fc2e488` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:50.744435 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03bfb88-260d-460b-b484-a6b6926152f2 None admin] RBAC: Authorizing `identity:create_grant(project_id=4e9987720930473da840579654311bc2, user_id=72b90d35087e4d7a80e9417082a1541f, role_id=0e661b5a67664692b933e5da9fc2e488)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:50.745900 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03bfb88-260d-460b-b484-a6b6926152f2 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:50.752658 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad69c457-ee02-4c58-a76a-6db38b13ec55 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:50.785483 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65512c63-44e4-477e-8eac-7aa294fad410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:50.786167 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65512c63-44e4-477e-8eac-7aa294fad410 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:50.786445 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65512c63-44e4-477e-8eac-7aa294fad410 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:50.786718 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65512c63-44e4-477e-8eac-7aa294fad410 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:50.794006 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65512c63-44e4-477e-8eac-7aa294fad410 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:50.795420 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65512c63-44e4-477e-8eac-7aa294fad410 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:50.807130 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-65512c63-44e4-477e-8eac-7aa294fad410 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 15 07:20:50.807130 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:50.807130 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:50.807130 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:50.807130 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:50.807130 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:50.807130 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:50.807130 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:50.807130 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:50.807130 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:50.807130 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:50.807130 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:50.807130 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 07:20:50.807130 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 07:20:50.807130 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 07:20:50.807130 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 07:20:50.808336 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:50.808336 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:50.808336 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 07:20:50.808336 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 07:20:50.808336 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 07:20:50.808336 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 07:20:50.808336 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:50.808336 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:50.808336 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:50.808336 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:50.808336 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:50.808336 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:50.808336 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:50.808336 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:50.808336 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:50.808336 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:50.808336 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:50.808336 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:50.809873 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:50.809873 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:50.809873 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 07:20:50.809873 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 07:20:50.809873 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 07:20:50.809873 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 07:20:50.809873 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 07:20:50.809873 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 07:20:50.809873 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 15 07:20:50.809873 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:20:50.810727 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 312/628] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 15 07:20:50 2022] GET /identity/v3/users/glance => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 07:20:50.811573 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65512c63-44e4-477e-8eac-7aa294fad410 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:50.818910 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 317/629] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Sep 15 07:20:50 2022] PUT /identity/v3/projects/4e9987720930473da840579654311bc2/users/72b90d35087e4d7a80e9417082a1541f/roles/0e661b5a67664692b933e5da9fc2e488 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:20:50.820466 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedeba06-fd7c-466d-91af-8a8d12e75264 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:50.820998 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedeba06-fd7c-466d-91af-8a8d12e75264 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:50.821501 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedeba06-fd7c-466d-91af-8a8d12e75264 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:50.823115 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 318/630] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:20:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:20:50.824117 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedeba06-fd7c-466d-91af-8a8d12e75264 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:50.926146 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:50.933655 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:50.939761 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65512c63-44e4-477e-8eac-7aa294fad410 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:50.958216 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedeba06-fd7c-466d-91af-8a8d12e75264 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:50.979977 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4319e55c-15b4-43e3-854c-46c6bb07332f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:50.981122 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4319e55c-15b4-43e3-854c-46c6bb07332f None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:50.981661 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4319e55c-15b4-43e3-854c-46c6bb07332f None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:50.982172 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4319e55c-15b4-43e3-854c-46c6bb07332f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:50.994327 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4319e55c-15b4-43e3-854c-46c6bb07332f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:20:50.998636 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e017376a-8dd9-47e7-993a-03a90ec41366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:50.999815 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e017376a-8dd9-47e7-993a-03a90ec41366 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:51.000217 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e017376a-8dd9-47e7-993a-03a90ec41366 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:51.000541 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e017376a-8dd9-47e7-993a-03a90ec41366 None admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:51.001733 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e017376a-8dd9-47e7-993a-03a90ec41366 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:51.002365 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e017376a-8dd9-47e7-993a-03a90ec41366 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:51.003862 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-4319e55c-15b4-43e3-854c-46c6bb07332f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 07:20:51.003862 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:51.003862 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:51.003862 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:51.003862 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:51.003862 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:51.003862 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:51.003862 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:51.003862 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:51.003862 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:51.003862 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:51.003862 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:51.003862 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:20:51.003862 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:20:51.003862 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:20:51.003862 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:20:51.004850 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:20:51.004850 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:20:51.004850 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:20:51.004850 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:20:51.004850 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:51.004850 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:51.004850 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:51.004850 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:51.004850 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:51.004850 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:51.004850 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:51.004850 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:51.004850 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:51.004850 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:51.004850 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:51.004850 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:51.005745 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:51.005745 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:51.005745 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:51.005745 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:51.005745 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:20:51.005745 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:20:51.005745 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:20:51.005745 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:20:51.005745 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:20:51.005745 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:20:51.005745 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 07:20:51.005745 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:20:51.006406 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e017376a-8dd9-47e7-993a-03a90ec41366 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:51.008369 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 313/631] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Sep 15 07:20:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:51.010025 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd2f30-8d6b-4c08-b821-8fe3406ef06f None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:51.010884 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd2f30-8d6b-4c08-b821-8fe3406ef06f None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:51.011539 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd2f30-8d6b-4c08-b821-8fe3406ef06f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:51.029708 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 319/632] 10.222.0.42 () {56 vars in 1137 bytes} [Thu Sep 15 07:20:50 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:51.031295 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e017376a-8dd9-47e7-993a-03a90ec41366 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:51.039822 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-6bbd2f30-8d6b-4c08-b821-8fe3406ef06f None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:20:51.040997 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-6bbd2f30-8d6b-4c08-b821-8fe3406ef06f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:20:51.127980 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 314/633] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:20:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 07:20:51.129190 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbd2f30-8d6b-4c08-b821-8fe3406ef06f None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:51.179730 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e017376a-8dd9-47e7-993a-03a90ec41366 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:51.217462 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1c727f-e9c0-4477-aae7-52b91a28c2f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:51.218140 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c727f-e9c0-4477-aae7-52b91a28c2f6 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:51.218607 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c727f-e9c0-4477-aae7-52b91a28c2f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:51.218893 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c727f-e9c0-4477-aae7-52b91a28c2f6 None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:51.219561 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1c727f-e9c0-4477-aae7-52b91a28c2f6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:51.219870 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1c727f-e9c0-4477-aae7-52b91a28c2f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:51.221237 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1c727f-e9c0-4477-aae7-52b91a28c2f6 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:51.230927 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 320/634] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Sep 15 07:20:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:51.236506 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1c727f-e9c0-4477-aae7-52b91a28c2f6 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:51.255899 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:51.265949 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:51.301078 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbd2f30-8d6b-4c08-b821-8fe3406ef06f None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:51.386388 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7026d7-3ced-41f1-b1de-29381ad5ccbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:51.386933 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7026d7-3ced-41f1-b1de-29381ad5ccbf None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:51.387211 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7026d7-3ced-41f1-b1de-29381ad5ccbf None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:51.387488 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7026d7-3ced-41f1-b1de-29381ad5ccbf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:51.405441 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1c727f-e9c0-4477-aae7-52b91a28c2f6 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:51.460975 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780e25b6-88d2-452c-86fd-c9037cecea50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:51.461637 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780e25b6-88d2-452c-86fd-c9037cecea50 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:51.462453 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780e25b6-88d2-452c-86fd-c9037cecea50 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:51.462733 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780e25b6-88d2-452c-86fd-c9037cecea50 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:51.477587 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780e25b6-88d2-452c-86fd-c9037cecea50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:51.479116 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780e25b6-88d2-452c-86fd-c9037cecea50 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:51.495032 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7026d7-3ced-41f1-b1de-29381ad5ccbf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:51.495719 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-780e25b6-88d2-452c-86fd-c9037cecea50 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:20:51.495719 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:51.495719 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:51.495719 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:51.495719 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:51.495719 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:51.495719 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:51.495719 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:51.495719 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:51.495719 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:51.495719 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:51.495719 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:51.495719 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 07:20:51.495719 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 07:20:51.495719 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 07:20:51.495719 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 07:20:51.496605 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:51.496605 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:51.496605 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:51.496605 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:51.496605 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:51.496605 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:51.496605 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:51.496605 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:51.496605 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:51.496605 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:51.496605 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:51.496605 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:51.496605 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:51.496605 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:51.496605 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:51.496605 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:51.496605 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 07:20:51.498184 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 07:20:51.498184 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:20:51.498184 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:20:51.498351 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7026d7-3ced-41f1-b1de-29381ad5ccbf None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:51.498434 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 321/635] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:20:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:51.498434 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e67e720-8a2f-4e6b-916b-8a2b7f3092f3 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:51.498565 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e67e720-8a2f-4e6b-916b-8a2b7f3092f3 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:51.499151 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e67e720-8a2f-4e6b-916b-8a2b7f3092f3 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:51.500935 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 322/636] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:20:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:20:51.501663 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e67e720-8a2f-4e6b-916b-8a2b7f3092f3 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:51.570940 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:51.584202 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:51.605195 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 315/637] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 15 07:20:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 07:20:51.605918 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7026d7-3ced-41f1-b1de-29381ad5ccbf None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:51.619314 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e67e720-8a2f-4e6b-916b-8a2b7f3092f3 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:51.656377 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f658a47-a194-47f0-b3e9-ef5a7dff344a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:51.657088 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f658a47-a194-47f0-b3e9-ef5a7dff344a None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:51.657368 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f658a47-a194-47f0-b3e9-ef5a7dff344a None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:51.657677 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f658a47-a194-47f0-b3e9-ef5a7dff344a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:51.667861 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f658a47-a194-47f0-b3e9-ef5a7dff344a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:20:51.673845 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-4f658a47-a194-47f0-b3e9-ef5a7dff344a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 07:20:51.673845 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:51.673845 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:51.673845 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:51.673845 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:51.673845 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:51.673845 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:51.673845 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:51.673845 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:51.673845 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:51.673845 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:51.673845 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:51.673845 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:20:51.673845 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:20:51.673845 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:20:51.673845 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:20:51.674815 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:20:51.674815 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:20:51.674815 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:20:51.674815 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:20:51.674815 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:51.674815 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:51.674815 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:51.674815 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:51.674815 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:51.674815 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:51.674815 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:51.674815 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:51.674815 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:51.674815 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:51.674815 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:51.674815 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:51.675669 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:51.675669 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:51.675669 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:51.675669 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:51.675669 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:20:51.675669 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:20:51.675669 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:20:51.675669 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:20:51.675669 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:20:51.675669 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:20:51.675669 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 07:20:51.675669 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:20:51.675669 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 323/638] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Sep 15 07:20:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:51.676351 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52add515-afc3-49b5-83a8-fc24968d51f1 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:51.676504 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52add515-afc3-49b5-83a8-fc24968d51f1 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:51.677213 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52add515-afc3-49b5-83a8-fc24968d51f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:51.697950 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-52add515-afc3-49b5-83a8-fc24968d51f1 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:20:51.699186 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-52add515-afc3-49b5-83a8-fc24968d51f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:20:51.724737 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7026d7-3ced-41f1-b1de-29381ad5ccbf None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:51.762751 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f328bbed-4624-4b23-af29-9e0528727117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:51.763466 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f328bbed-4624-4b23-af29-9e0528727117 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:51.763747 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f328bbed-4624-4b23-af29-9e0528727117 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:51.764157 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f328bbed-4624-4b23-af29-9e0528727117 None admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:51.764768 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f328bbed-4624-4b23-af29-9e0528727117 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:51.765195 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f328bbed-4624-4b23-af29-9e0528727117 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:51.766591 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f328bbed-4624-4b23-af29-9e0528727117 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:51.777814 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 324/639] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:20:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 07:20:51.777814 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52add515-afc3-49b5-83a8-fc24968d51f1 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:51.788940 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 316/640] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 15 07:20:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:51.790001 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f328bbed-4624-4b23-af29-9e0528727117 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:51.840341 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:51.845677 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:51.907219 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52add515-afc3-49b5-83a8-fc24968d51f1 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:51.916571 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f328bbed-4624-4b23-af29-9e0528727117 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:51.957384 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47327e84-a761-46ac-b35a-6ab16c6916c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:51.958661 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47327e84-a761-46ac-b35a-6ab16c6916c6 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:51.959338 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47327e84-a761-46ac-b35a-6ab16c6916c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:51.959789 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47327e84-a761-46ac-b35a-6ab16c6916c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:51.966066 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5446e487-6537-404f-9e11-11214b07eccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:51.966770 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5446e487-6537-404f-9e11-11214b07eccb None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:51.967144 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5446e487-6537-404f-9e11-11214b07eccb None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:51.967507 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5446e487-6537-404f-9e11-11214b07eccb None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:51.968501 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5446e487-6537-404f-9e11-11214b07eccb None admin] RBAC: Adding query filter params (name=service) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:51.969027 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5446e487-6537-404f-9e11-11214b07eccb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:51.970514 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5446e487-6537-404f-9e11-11214b07eccb None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:51.974560 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47327e84-a761-46ac-b35a-6ab16c6916c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:51.976733 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47327e84-a761-46ac-b35a-6ab16c6916c6 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:51.984789 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 317/641] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Sep 15 07:20:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:51.984789 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5446e487-6537-404f-9e11-11214b07eccb None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:52.003309 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-47327e84-a761-46ac-b35a-6ab16c6916c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:20:52.003309 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:52.003309 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:52.003309 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:52.003309 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:52.003309 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:52.003309 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:52.003309 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:52.003309 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:52.003309 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:52.003309 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:52.003309 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:52.003309 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 07:20:52.003309 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 07:20:52.003309 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 07:20:52.003309 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 07:20:52.004467 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:52.004467 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:52.004467 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:52.004467 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:52.004467 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:52.004467 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:52.004467 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:52.004467 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:52.004467 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:52.004467 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:52.004467 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:52.004467 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:52.004467 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:52.004467 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:52.004467 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:52.004467 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:52.004467 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 07:20:52.005590 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 07:20:52.005590 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:20:52.005590 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:20:52.007368 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 325/642] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:20:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:52.008330 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47327e84-a761-46ac-b35a-6ab16c6916c6 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:52.111895 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5446e487-6537-404f-9e11-11214b07eccb None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:52.114959 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:52.145152 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47327e84-a761-46ac-b35a-6ab16c6916c6 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:52.147610 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:52.151126 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9872a5-ef1b-4863-be78-07fb08b85be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:52.151773 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9872a5-ef1b-4863-be78-07fb08b85be1 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:52.152050 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9872a5-ef1b-4863-be78-07fb08b85be1 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:52.152738 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9872a5-ef1b-4863-be78-07fb08b85be1 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:52.162069 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9872a5-ef1b-4863-be78-07fb08b85be1 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:52.163782 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9872a5-ef1b-4863-be78-07fb08b85be1 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:52.175844 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-3c9872a5-ef1b-4863-be78-07fb08b85be1 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 15 07:20:52.175844 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:52.175844 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:52.175844 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:52.175844 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:52.175844 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:52.175844 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:52.175844 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:52.175844 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:52.175844 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:52.175844 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:52.175844 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:52.175844 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 07:20:52.175844 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 07:20:52.175844 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 07:20:52.175844 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 07:20:52.176871 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:52.176871 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:52.176871 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 07:20:52.176871 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 07:20:52.176871 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 07:20:52.176871 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 07:20:52.176871 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:52.176871 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:52.176871 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:52.176871 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:52.176871 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:52.176871 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:52.176871 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:52.176871 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:52.176871 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:52.176871 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:52.176871 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:52.176871 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:52.179076 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:52.179076 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:52.179076 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 07:20:52.179076 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 07:20:52.179076 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 07:20:52.179076 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 07:20:52.179076 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 07:20:52.179076 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 07:20:52.179076 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 15 07:20:52.179076 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:20:52.179076 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 318/643] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 15 07:20:51 2022] GET /identity/v3/users/cinder => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:52.179076 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9872a5-ef1b-4863-be78-07fb08b85be1 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:52.186340 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641aea36-2a62-4e8a-9d7b-186890c468df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:52.187665 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641aea36-2a62-4e8a-9d7b-186890c468df None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:52.187985 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641aea36-2a62-4e8a-9d7b-186890c468df None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:52.188229 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641aea36-2a62-4e8a-9d7b-186890c468df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:52.199083 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641aea36-2a62-4e8a-9d7b-186890c468df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:52.200506 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641aea36-2a62-4e8a-9d7b-186890c468df None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:52.215292 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-641aea36-2a62-4e8a-9d7b-186890c468df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:20:52.215292 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:52.215292 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:52.215292 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:52.215292 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:52.215292 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:52.215292 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:52.215292 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:52.215292 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:52.215292 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:52.215292 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:52.215292 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:52.215292 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 07:20:52.215292 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 07:20:52.215292 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 07:20:52.215292 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 07:20:52.216245 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:52.216245 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:52.216245 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:52.216245 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:52.216245 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:52.216245 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:52.216245 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:52.216245 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:52.216245 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:52.216245 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:52.216245 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:52.216245 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:52.216245 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:52.216245 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:52.216245 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:52.216245 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:52.216245 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 07:20:52.217293 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 07:20:52.217293 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:20:52.217293 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:20:52.217293 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 326/644] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:20:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:52.217293 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641aea36-2a62-4e8a-9d7b-186890c468df None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:52.290334 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9872a5-ef1b-4863-be78-07fb08b85be1 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:52.324332 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641aea36-2a62-4e8a-9d7b-186890c468df None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:52.325920 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa51913a-6abb-4b20-9de6-c63efd89a91c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:52.326529 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa51913a-6abb-4b20-9de6-c63efd89a91c None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:52.327376 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa51913a-6abb-4b20-9de6-c63efd89a91c None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:52.327376 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa51913a-6abb-4b20-9de6-c63efd89a91c None admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:52.328216 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa51913a-6abb-4b20-9de6-c63efd89a91c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:52.328524 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa51913a-6abb-4b20-9de6-c63efd89a91c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:52.329895 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa51913a-6abb-4b20-9de6-c63efd89a91c None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:52.347159 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 319/645] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 15 07:20:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:52.347159 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa51913a-6abb-4b20-9de6-c63efd89a91c None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:52.365418 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8568b061-8fca-4993-a609-5d408b727cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:52.366080 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8568b061-8fca-4993-a609-5d408b727cea None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:52.366358 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8568b061-8fca-4993-a609-5d408b727cea None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:52.366629 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8568b061-8fca-4993-a609-5d408b727cea None admin] PATH_INFO: `/v3/roles/service` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:52.372726 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:52.377661 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8568b061-8fca-4993-a609-5d408b727cea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:20:52.383421 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-8568b061-8fca-4993-a609-5d408b727cea None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 07:20:52.383421 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:52.383421 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:52.383421 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:52.383421 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:52.383421 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:52.383421 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:52.383421 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:52.383421 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:52.383421 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:52.383421 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:52.383421 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:52.383421 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:20:52.383421 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:20:52.383421 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:20:52.383421 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:20:52.384353 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:20:52.384353 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:20:52.384353 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:20:52.384353 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:20:52.384353 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:52.384353 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:52.384353 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:52.384353 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:52.384353 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:52.384353 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:52.384353 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:52.384353 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:52.384353 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:52.384353 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:52.384353 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:52.384353 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:52.385201 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:52.385201 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:52.385201 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:52.385201 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:52.385201 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:20:52.385201 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:20:52.385201 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:20:52.385201 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:20:52.385201 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:20:52.385201 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:20:52.385201 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 07:20:52.385201 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:20:52.385201 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 327/646] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Sep 15 07:20:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:52.385939 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8568b061-8fca-4993-a609-5d408b727cea None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:52.404666 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:52.460013 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa51913a-6abb-4b20-9de6-c63efd89a91c None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:52.493031 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8568b061-8fca-4993-a609-5d408b727cea None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:52.496221 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09608c6-2dff-4b9a-afb2-132f54eb800c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:52.496840 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09608c6-2dff-4b9a-afb2-132f54eb800c None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:52.497123 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09608c6-2dff-4b9a-afb2-132f54eb800c None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:52.497397 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09608c6-2dff-4b9a-afb2-132f54eb800c None admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:52.498158 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09608c6-2dff-4b9a-afb2-132f54eb800c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:52.498469 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09608c6-2dff-4b9a-afb2-132f54eb800c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:52.500952 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09608c6-2dff-4b9a-afb2-132f54eb800c None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:52.520669 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 320/647] 10.222.0.42 () {56 vars in 1137 bytes} [Thu Sep 15 07:20:52 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:52.521378 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09608c6-2dff-4b9a-afb2-132f54eb800c None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:52.533277 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bb77ab-f5f2-49bc-a928-f7d9cb666050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:52.533918 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bb77ab-f5f2-49bc-a928-f7d9cb666050 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:52.534194 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bb77ab-f5f2-49bc-a928-f7d9cb666050 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:52.534466 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bb77ab-f5f2-49bc-a928-f7d9cb666050 None admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:52.535135 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bb77ab-f5f2-49bc-a928-f7d9cb666050 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:52.535440 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bb77ab-f5f2-49bc-a928-f7d9cb666050 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:52.536790 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bb77ab-f5f2-49bc-a928-f7d9cb666050 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:52.549385 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 328/648] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 15 07:20:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:52.550084 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bb77ab-f5f2-49bc-a928-f7d9cb666050 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:52.613906 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:52.627907 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:52.634801 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09608c6-2dff-4b9a-afb2-132f54eb800c None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:52.650689 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bb77ab-f5f2-49bc-a928-f7d9cb666050 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:52.675135 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca7bad9-0d00-4544-8800-55259137f445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:52.675775 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca7bad9-0d00-4544-8800-55259137f445 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:52.676064 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca7bad9-0d00-4544-8800-55259137f445 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:52.676348 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca7bad9-0d00-4544-8800-55259137f445 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:52.683087 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca7bad9-0d00-4544-8800-55259137f445 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:52.684738 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca7bad9-0d00-4544-8800-55259137f445 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:52.688310 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5773f9f-bfde-428c-8bd6-7dab5aab8375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:52.688927 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5773f9f-bfde-428c-8bd6-7dab5aab8375 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:52.689205 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5773f9f-bfde-428c-8bd6-7dab5aab8375 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:52.689475 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5773f9f-bfde-428c-8bd6-7dab5aab8375 None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:52.690164 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5773f9f-bfde-428c-8bd6-7dab5aab8375 None admin] RBAC: Adding query filter params (name=service) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:52.690466 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5773f9f-bfde-428c-8bd6-7dab5aab8375 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:52.691814 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5773f9f-bfde-428c-8bd6-7dab5aab8375 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:52.696543 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-5ca7bad9-0d00-4544-8800-55259137f445 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 07:20:52.696543 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:52.696543 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:52.696543 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:52.696543 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:52.696543 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:52.696543 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:52.696543 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:52.696543 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:52.696543 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:52.696543 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:52.696543 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:52.696543 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 07:20:52.696543 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 07:20:52.696543 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 07:20:52.697410 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 07:20:52.697410 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:52.697410 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:52.697410 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:52.697410 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:52.697410 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:52.697410 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:52.697410 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:52.697410 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:52.697410 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:52.697410 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:52.697410 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:52.697410 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:52.697410 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:52.697410 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:52.697410 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:52.697410 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:52.698586 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 07:20:52.698586 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 07:20:52.698586 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 07:20:52.698586 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 07:20:52.698586 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 07:20:52.698586 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 07:20:52.698586 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 07:20:52.698586 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:20:52.698586 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 321/649] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 15 07:20:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:52.698586 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca7bad9-0d00-4544-8800-55259137f445 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:52.705961 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 329/650] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Sep 15 07:20:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:52.707929 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5773f9f-bfde-428c-8bd6-7dab5aab8375 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:52.803611 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca7bad9-0d00-4544-8800-55259137f445 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:52.820904 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5773f9f-bfde-428c-8bd6-7dab5aab8375 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:52.840867 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f6ab33-e103-42e4-8ef5-45dbe369d12c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:52.841468 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f6ab33-e103-42e4-8ef5-45dbe369d12c None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:52.841768 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f6ab33-e103-42e4-8ef5-45dbe369d12c None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:52.842040 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f6ab33-e103-42e4-8ef5-45dbe369d12c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:52.853389 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:52.857709 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d44026-03d9-4a6b-bc32-c1ca87096df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:52.858348 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d44026-03d9-4a6b-bc32-c1ca87096df4 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:52.858631 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d44026-03d9-4a6b-bc32-c1ca87096df4 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:52.859433 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d44026-03d9-4a6b-bc32-c1ca87096df4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:52.868169 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d44026-03d9-4a6b-bc32-c1ca87096df4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:52.869795 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d44026-03d9-4a6b-bc32-c1ca87096df4 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:52.871630 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:52.881391 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-78d44026-03d9-4a6b-bc32-c1ca87096df4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 15 07:20:52.881391 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:52.881391 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:52.881391 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:52.881391 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:52.881391 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:52.881391 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:52.881391 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:52.881391 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:52.881391 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:52.881391 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:52.881391 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:52.881391 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 07:20:52.881391 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 07:20:52.881391 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 07:20:52.881391 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 07:20:52.882514 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:52.882514 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:52.882514 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 07:20:52.882514 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 07:20:52.882514 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 07:20:52.882514 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 07:20:52.882514 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:52.882514 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:52.882514 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:52.882514 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:52.882514 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:52.882514 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:52.882514 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:52.882514 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:52.882514 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:52.882514 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:52.882514 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:52.882514 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:52.883430 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:52.883430 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:52.883430 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 07:20:52.883430 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 07:20:52.883430 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 07:20:52.883430 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 07:20:52.883430 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 07:20:52.883430 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 07:20:52.883430 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 15 07:20:52.883430 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:20:52.883430 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 330/651] 10.222.0.42 () {56 vars in 1093 bytes} [Thu Sep 15 07:20:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:52.883430 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d44026-03d9-4a6b-bc32-c1ca87096df4 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:52.919106 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f6ab33-e103-42e4-8ef5-45dbe369d12c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:52.920768 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f6ab33-e103-42e4-8ef5-45dbe369d12c None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:52.994515 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d44026-03d9-4a6b-bc32-c1ca87096df4 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:53.029055 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156a7010-350e-468d-93f2-5b86121a2645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:53.029684 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156a7010-350e-468d-93f2-5b86121a2645 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:53.029961 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156a7010-350e-468d-93f2-5b86121a2645 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:53.030237 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156a7010-350e-468d-93f2-5b86121a2645 None admin] PATH_INFO: `/v3/projects` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:53.030982 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156a7010-350e-468d-93f2-5b86121a2645 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:53.031318 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156a7010-350e-468d-93f2-5b86121a2645 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:53.032827 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156a7010-350e-468d-93f2-5b86121a2645 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:53.034715 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 322/652] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 15 07:20:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 07:20:53.035508 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f6ab33-e103-42e4-8ef5-45dbe369d12c None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:53.050487 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 331/653] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 15 07:20:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:53.055881 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156a7010-350e-468d-93f2-5b86121a2645 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:53.081102 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:53.118821 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:53.147444 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f6ab33-e103-42e4-8ef5-45dbe369d12c None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:53.163809 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156a7010-350e-468d-93f2-5b86121a2645 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:53.183729 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e92951-92f7-4a3e-b6d1-0c7d6495dfb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:53.184816 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e92951-92f7-4a3e-b6d1-0c7d6495dfb9 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:53.185191 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e92951-92f7-4a3e-b6d1-0c7d6495dfb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:53.185519 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e92951-92f7-4a3e-b6d1-0c7d6495dfb9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:53.197133 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e92951-92f7-4a3e-b6d1-0c7d6495dfb9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:53.198279 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bffc11-5932-4ae1-bb00-e10e2a13a6fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:53.198606 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e92951-92f7-4a3e-b6d1-0c7d6495dfb9 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:53.198886 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bffc11-5932-4ae1-bb00-e10e2a13a6fe None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:53.199165 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bffc11-5932-4ae1-bb00-e10e2a13a6fe None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:53.199435 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bffc11-5932-4ae1-bb00-e10e2a13a6fe None admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:53.200150 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bffc11-5932-4ae1-bb00-e10e2a13a6fe None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:53.200518 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bffc11-5932-4ae1-bb00-e10e2a13a6fe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:53.201975 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bffc11-5932-4ae1-bb00-e10e2a13a6fe None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:53.213942 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-49e92951-92f7-4a3e-b6d1-0c7d6495dfb9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:20:53.213942 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:53.213942 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:53.213942 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:53.213942 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:53.213942 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:53.213942 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:53.213942 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:53.213942 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:53.213942 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:53.213942 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:53.213942 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:53.213942 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 07:20:53.213942 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 07:20:53.213942 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 07:20:53.213942 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 07:20:53.215031 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:53.215031 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:53.215031 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:53.215031 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:53.215031 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:53.215031 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:53.215031 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:53.215031 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:53.215031 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:53.215031 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:53.215031 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:53.215031 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:53.215031 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:53.215031 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:53.215031 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:53.215031 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:53.215031 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 07:20:53.221311 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 07:20:53.221311 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:20:53.221311 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:20:53.221311 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 323/654] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:20:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:53.221311 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e92951-92f7-4a3e-b6d1-0c7d6495dfb9 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:53.221760 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 332/655] 10.222.0.42 () {56 vars in 1133 bytes} [Thu Sep 15 07:20:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:53.221760 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bffc11-5932-4ae1-bb00-e10e2a13a6fe None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:53.311133 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:53.320353 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e92951-92f7-4a3e-b6d1-0c7d6495dfb9 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:53.322385 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bffc11-5932-4ae1-bb00-e10e2a13a6fe None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:53.338265 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:53.354648 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa0ff5c-3399-4d2b-b138-d7c96592e5a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:53.362648 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa0ff5c-3399-4d2b-b138-d7c96592e5a8 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:53.362648 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa0ff5c-3399-4d2b-b138-d7c96592e5a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:53.362648 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa0ff5c-3399-4d2b-b138-d7c96592e5a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:53.362932 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d421b35b-c2ca-4bb1-aaa3-696a1b96fe62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:53.362932 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d421b35b-c2ca-4bb1-aaa3-696a1b96fe62 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:53.362932 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d421b35b-c2ca-4bb1-aaa3-696a1b96fe62 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:53.362932 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d421b35b-c2ca-4bb1-aaa3-696a1b96fe62 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:53.363234 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d421b35b-c2ca-4bb1-aaa3-696a1b96fe62 None admin] RBAC: Adding query filter params (role.id=0e661b5a67664692b933e5da9fc2e488, scope.project.id=4e9987720930473da840579654311bc2, user.id=37438b032a1f46d3bdf7121ad0cffbc3) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:53.363234 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d421b35b-c2ca-4bb1-aaa3-696a1b96fe62 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:53.363234 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d421b35b-c2ca-4bb1-aaa3-696a1b96fe62 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:53.366619 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 333/656] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 15 07:20:53 2022] GET /identity/v3/role_assignments?user.id=37438b032a1f46d3bdf7121ad0cffbc3&scope.project.id=4e9987720930473da840579654311bc2&role.id=0e661b5a67664692b933e5da9fc2e488 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:53.366820 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa0ff5c-3399-4d2b-b138-d7c96592e5a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:53.367849 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df374f7e-6c60-4829-8ecb-352fd1858c46 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:53.368135 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df374f7e-6c60-4829-8ecb-352fd1858c46 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:53.368279 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa0ff5c-3399-4d2b-b138-d7c96592e5a8 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:53.368405 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df374f7e-6c60-4829-8ecb-352fd1858c46 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:53.369783 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 334/657] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:20:53 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:20:53.370369 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df374f7e-6c60-4829-8ecb-352fd1858c46 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:53.394540 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-faa0ff5c-3399-4d2b-b138-d7c96592e5a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:20:53.394540 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:53.394540 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:53.394540 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:53.394540 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:53.394540 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:53.394540 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:53.394540 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:53.394540 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:53.394540 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:53.394540 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:53.394540 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:53.394540 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 07:20:53.394540 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 07:20:53.394540 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 07:20:53.394540 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 07:20:53.395673 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:53.395673 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:53.395673 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:53.395673 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:53.395673 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:53.395673 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:53.395673 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:53.395673 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:53.395673 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:53.395673 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:53.395673 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:53.395673 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:53.395673 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:53.395673 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:53.395673 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:53.395673 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:53.395673 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 07:20:53.396812 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 07:20:53.396812 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:20:53.396812 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:20:53.397168 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 324/658] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:20:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:53.398263 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa0ff5c-3399-4d2b-b138-d7c96592e5a8 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:53.509651 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df374f7e-6c60-4829-8ecb-352fd1858c46 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:53.515898 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa0ff5c-3399-4d2b-b138-d7c96592e5a8 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:53.553901 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43faeca-1ee2-461d-83bd-82536e6bcec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:53.555321 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43faeca-1ee2-461d-83bd-82536e6bcec0 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:53.555321 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43faeca-1ee2-461d-83bd-82536e6bcec0 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:53.555321 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43faeca-1ee2-461d-83bd-82536e6bcec0 None admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:53.555650 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140bc5c7-d6fb-463e-9875-996b8970a008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:53.555759 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43faeca-1ee2-461d-83bd-82536e6bcec0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:53.555994 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43faeca-1ee2-461d-83bd-82536e6bcec0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:53.556265 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140bc5c7-d6fb-463e-9875-996b8970a008 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:53.558031 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140bc5c7-d6fb-463e-9875-996b8970a008 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:53.558031 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140bc5c7-d6fb-463e-9875-996b8970a008 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:53.558216 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43faeca-1ee2-461d-83bd-82536e6bcec0 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:53.571234 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 335/659] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 15 07:20:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:53.572529 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6da811-52f6-40f2-93c1-8a71b0a30f1d None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:53.572822 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6da811-52f6-40f2-93c1-8a71b0a30f1d None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:53.573101 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6da811-52f6-40f2-93c1-8a71b0a30f1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:53.579430 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:53.595273 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-9d6da811-52f6-40f2-93c1-8a71b0a30f1d None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:20:53.596425 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-9d6da811-52f6-40f2-93c1-8a71b0a30f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:20:53.633919 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:53.647646 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140bc5c7-d6fb-463e-9875-996b8970a008 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:53.649047 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140bc5c7-d6fb-463e-9875-996b8970a008 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:53.673603 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 336/660] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:20:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 07:20:53.673603 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6da811-52f6-40f2-93c1-8a71b0a30f1d None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:53.792882 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6da811-52f6-40f2-93c1-8a71b0a30f1d None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:53.794867 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 325/661] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 15 07:20:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 07:20:53.795853 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140bc5c7-d6fb-463e-9875-996b8970a008 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:53.840798 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a8db66-9531-4027-862d-360775444260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:53.841320 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a8db66-9531-4027-862d-360775444260 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:53.841610 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a8db66-9531-4027-862d-360775444260 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:53.841891 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a8db66-9531-4027-862d-360775444260 None admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:53.842525 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a8db66-9531-4027-862d-360775444260 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:53.842825 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a8db66-9531-4027-862d-360775444260 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:53.845140 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a8db66-9531-4027-862d-360775444260 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:53.857197 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 337/662] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 15 07:20:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:53.858206 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a8db66-9531-4027-862d-360775444260 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:53.865233 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:53.892236 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:53.941324 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140bc5c7-d6fb-463e-9875-996b8970a008 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:53.977300 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a8db66-9531-4027-862d-360775444260 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:53.990397 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fcf0e8-2999-4f16-8569-b328b4e55e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:53.991170 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fcf0e8-2999-4f16-8569-b328b4e55e49 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:53.991530 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fcf0e8-2999-4f16-8569-b328b4e55e49 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:53.991805 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fcf0e8-2999-4f16-8569-b328b4e55e49 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:53.999161 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fcf0e8-2999-4f16-8569-b328b4e55e49 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:54.001440 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fcf0e8-2999-4f16-8569-b328b4e55e49 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:54.011829 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433dea04-19a6-4cf3-bc96-14c13708bb74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:54.012642 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433dea04-19a6-4cf3-bc96-14c13708bb74 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:54.013063 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433dea04-19a6-4cf3-bc96-14c13708bb74 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:54.013251 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-90fcf0e8-2999-4f16-8569-b328b4e55e49 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 15 07:20:54.013251 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:54.013251 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:54.013251 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:54.013251 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:54.013251 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:54.013251 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:54.013251 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:54.013251 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:54.013251 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:54.013251 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:54.013251 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:54.013251 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 07:20:54.013251 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 07:20:54.013251 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 07:20:54.013251 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 07:20:54.014310 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:54.014310 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:54.014310 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 07:20:54.014310 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 07:20:54.014310 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 07:20:54.014310 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 07:20:54.014310 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:54.014310 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:54.014310 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:54.014310 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:54.014310 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:54.014310 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:54.014310 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:54.014310 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:54.014310 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:54.014310 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:54.014310 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:54.014310 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:54.015395 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:54.015395 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:54.015395 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 07:20:54.015395 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 07:20:54.015395 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 07:20:54.015395 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 07:20:54.015395 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 07:20:54.015395 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 07:20:54.015395 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 15 07:20:54.015395 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:20:54.015395 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 326/663] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Sep 15 07:20:53 2022] GET /identity/v3/users/neutron => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:54.016076 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fcf0e8-2999-4f16-8569-b328b4e55e49 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:54.016461 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433dea04-19a6-4cf3-bc96-14c13708bb74 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:54.032346 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433dea04-19a6-4cf3-bc96-14c13708bb74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:54.034050 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433dea04-19a6-4cf3-bc96-14c13708bb74 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:54.049096 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-433dea04-19a6-4cf3-bc96-14c13708bb74 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:20:54.049096 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:54.049096 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:54.049096 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:54.049096 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:54.049096 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:54.049096 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:54.049096 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:54.049096 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:54.049096 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:54.049096 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:54.049096 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:54.049096 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 07:20:54.049096 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 07:20:54.049096 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 07:20:54.049096 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 07:20:54.050043 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:54.050043 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:54.050043 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:54.050043 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:54.050043 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:54.050043 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:54.050043 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:54.050043 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:54.050043 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:54.050043 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:54.050043 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:54.050043 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:54.050043 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:54.050043 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:54.050043 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:54.050043 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:54.050043 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 07:20:54.050921 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 07:20:54.050921 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:20:54.050921 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:20:54.050921 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 338/664] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:20:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:54.051549 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433dea04-19a6-4cf3-bc96-14c13708bb74 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:54.126382 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fcf0e8-2999-4f16-8569-b328b4e55e49 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:54.128722 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:54.141058 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:54.166009 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433dea04-19a6-4cf3-bc96-14c13708bb74 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:54.183898 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406333c1-028e-4611-901a-3f1765a18d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:54.184890 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406333c1-028e-4611-901a-3f1765a18d4c None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:54.185248 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406333c1-028e-4611-901a-3f1765a18d4c None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:54.185603 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406333c1-028e-4611-901a-3f1765a18d4c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:54.197729 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406333c1-028e-4611-901a-3f1765a18d4c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:54.199230 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406333c1-028e-4611-901a-3f1765a18d4c None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:54.211954 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-406333c1-028e-4611-901a-3f1765a18d4c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 07:20:54.211954 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:54.211954 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:54.211954 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:54.211954 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:54.211954 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:54.211954 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:54.211954 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:54.211954 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:54.211954 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:54.211954 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:54.211954 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:54.211954 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 07:20:54.211954 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 07:20:54.211954 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 07:20:54.212878 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 07:20:54.212878 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:54.212878 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:54.212878 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:54.212878 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:54.212878 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:54.212878 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:54.212878 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:54.212878 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:54.212878 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:54.212878 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:54.212878 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:54.212878 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:54.212878 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:54.212878 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:54.212878 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:54.212878 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:54.218952 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 07:20:54.218952 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 07:20:54.218952 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 07:20:54.218952 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 07:20:54.218952 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 07:20:54.218952 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 07:20:54.218952 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 07:20:54.218952 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:20:54.219456 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2f12e8-6ed6-41be-96cf-3fa39c31ecd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:54.219456 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2f12e8-6ed6-41be-96cf-3fa39c31ecd3 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:54.219456 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2f12e8-6ed6-41be-96cf-3fa39c31ecd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:54.219456 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2f12e8-6ed6-41be-96cf-3fa39c31ecd3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:54.219785 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 327/665] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 15 07:20:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:54.219785 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406333c1-028e-4611-901a-3f1765a18d4c None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:54.231121 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2f12e8-6ed6-41be-96cf-3fa39c31ecd3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:20:54.236835 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-1f2f12e8-6ed6-41be-96cf-3fa39c31ecd3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 07:20:54.236835 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:54.236835 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:54.236835 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:54.236835 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:54.236835 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:54.236835 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:54.236835 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:54.236835 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:54.236835 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:54.236835 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:54.236835 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:54.236835 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:20:54.236835 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:20:54.236835 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:20:54.236835 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:20:54.238011 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:20:54.238011 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:20:54.238011 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:20:54.238011 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:20:54.238011 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:54.238011 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:54.238011 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:54.238011 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:54.238011 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:54.238011 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:54.238011 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:54.238011 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:54.238011 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:54.238011 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:54.238011 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:54.238011 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:54.239090 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:54.239090 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:54.239090 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:54.239090 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:54.239090 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:20:54.239090 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:20:54.239090 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:20:54.239090 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:20:54.239090 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:20:54.239090 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:20:54.239090 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 07:20:54.239090 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:20:54.242783 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 339/666] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Sep 15 07:20:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:54.243627 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2f12e8-6ed6-41be-96cf-3fa39c31ecd3 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:54.330775 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406333c1-028e-4611-901a-3f1765a18d4c None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:54.351095 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2f12e8-6ed6-41be-96cf-3fa39c31ecd3 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:54.368917 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bf9166-0666-44e4-925b-e5bb01e157cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:54.369524 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bf9166-0666-44e4-925b-e5bb01e157cf None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:54.369838 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bf9166-0666-44e4-925b-e5bb01e157cf None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:54.370119 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bf9166-0666-44e4-925b-e5bb01e157cf None admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:54.370845 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bf9166-0666-44e4-925b-e5bb01e157cf None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:54.371153 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bf9166-0666-44e4-925b-e5bb01e157cf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:54.372570 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bf9166-0666-44e4-925b-e5bb01e157cf None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:54.386876 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1613ebd7-3d12-4f83-b03c-91f20fdb8084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:54.387424 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1613ebd7-3d12-4f83-b03c-91f20fdb8084 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:54.387701 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1613ebd7-3d12-4f83-b03c-91f20fdb8084 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:54.387977 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1613ebd7-3d12-4f83-b03c-91f20fdb8084 None admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:54.388616 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1613ebd7-3d12-4f83-b03c-91f20fdb8084 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:54.388918 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1613ebd7-3d12-4f83-b03c-91f20fdb8084 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:54.391407 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1613ebd7-3d12-4f83-b03c-91f20fdb8084 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:54.392323 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 328/667] 10.222.0.42 () {56 vars in 1139 bytes} [Thu Sep 15 07:20:54 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:54.393417 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bf9166-0666-44e4-925b-e5bb01e157cf None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:54.409284 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 340/668] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 15 07:20:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:54.410242 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1613ebd7-3d12-4f83-b03c-91f20fdb8084 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:54.416362 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:54.425885 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:54.506396 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bf9166-0666-44e4-925b-e5bb01e157cf None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:54.519304 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1613ebd7-3d12-4f83-b03c-91f20fdb8084 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:54.553073 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7958984b-9ac8-46b3-b58e-8f8d9c5ba41c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:54.553752 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7958984b-9ac8-46b3-b58e-8f8d9c5ba41c None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:54.553989 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7958984b-9ac8-46b3-b58e-8f8d9c5ba41c None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:54.554323 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7958984b-9ac8-46b3-b58e-8f8d9c5ba41c None admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:54.554962 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7958984b-9ac8-46b3-b58e-8f8d9c5ba41c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:54.555260 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7958984b-9ac8-46b3-b58e-8f8d9c5ba41c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:54.556655 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7958984b-9ac8-46b3-b58e-8f8d9c5ba41c None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:54.558501 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8bb82d-48af-4e46-821f-cb810e3c369a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:54.559120 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8bb82d-48af-4e46-821f-cb810e3c369a None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:54.559400 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8bb82d-48af-4e46-821f-cb810e3c369a None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:54.559941 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8bb82d-48af-4e46-821f-cb810e3c369a None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:54.560595 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8bb82d-48af-4e46-821f-cb810e3c369a None admin] RBAC: Adding query filter params (name=service) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:54.560905 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8bb82d-48af-4e46-821f-cb810e3c369a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:54.562869 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8bb82d-48af-4e46-821f-cb810e3c369a None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:54.575125 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 329/669] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 15 07:20:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:54.575125 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7958984b-9ac8-46b3-b58e-8f8d9c5ba41c None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:54.575785 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 341/670] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Sep 15 07:20:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:54.575785 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8bb82d-48af-4e46-821f-cb810e3c369a None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:54.644616 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:54.663227 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:54.681378 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7958984b-9ac8-46b3-b58e-8f8d9c5ba41c None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:54.683641 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8bb82d-48af-4e46-821f-cb810e3c369a None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:54.717626 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5e847a-2b3c-4b26-a84c-db296ae84c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:54.719213 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3de07d2-ad0d-4679-831b-8a4fc41d0dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:54.719489 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5e847a-2b3c-4b26-a84c-db296ae84c80 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:54.719839 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3de07d2-ad0d-4679-831b-8a4fc41d0dcc None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:54.720116 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3de07d2-ad0d-4679-831b-8a4fc41d0dcc None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:54.720385 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3de07d2-ad0d-4679-831b-8a4fc41d0dcc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:54.720608 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5e847a-2b3c-4b26-a84c-db296ae84c80 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:54.721073 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5e847a-2b3c-4b26-a84c-db296ae84c80 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:54.727147 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3de07d2-ad0d-4679-831b-8a4fc41d0dcc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:54.728584 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3de07d2-ad0d-4679-831b-8a4fc41d0dcc None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:54.739995 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-b3de07d2-ad0d-4679-831b-8a4fc41d0dcc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 07:20:54.739995 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:54.739995 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:54.739995 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:54.739995 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:54.739995 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:54.739995 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:54.739995 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:54.739995 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:54.739995 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:54.739995 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:54.739995 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:54.739995 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 07:20:54.739995 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 07:20:54.739995 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 07:20:54.740911 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 07:20:54.740911 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:54.740911 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:54.740911 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:54.740911 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:54.740911 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:54.740911 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:54.740911 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:54.740911 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:54.740911 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:54.740911 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:54.740911 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:54.740911 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:54.740911 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:54.740911 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:54.740911 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:54.740911 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:54.741937 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 07:20:54.741937 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 07:20:54.741937 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 07:20:54.741937 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 07:20:54.741937 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 07:20:54.741937 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 07:20:54.741937 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 07:20:54.741937 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:20:54.745924 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 342/671] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 15 07:20:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:54.746790 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3de07d2-ad0d-4679-831b-8a4fc41d0dcc None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:54.793874 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5e847a-2b3c-4b26-a84c-db296ae84c80 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:54.795310 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5e847a-2b3c-4b26-a84c-db296ae84c80 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:54.861221 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3de07d2-ad0d-4679-831b-8a4fc41d0dcc None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:54.882826 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:54.885269 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:54.897107 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b853f52c-8066-4788-9b5f-09465b611b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:54.897107 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b853f52c-8066-4788-9b5f-09465b611b41 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:54.897107 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b853f52c-8066-4788-9b5f-09465b611b41 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:54.897377 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b853f52c-8066-4788-9b5f-09465b611b41 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:54.898153 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b853f52c-8066-4788-9b5f-09465b611b41 None admin] RBAC: Adding query filter params (role.id=0e661b5a67664692b933e5da9fc2e488, scope.project.id=4e9987720930473da840579654311bc2, user.id=3ecf6642b7b84d3c9756bb41ab3349d3) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:54.898465 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b853f52c-8066-4788-9b5f-09465b611b41 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:54.899883 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b853f52c-8066-4788-9b5f-09465b611b41 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:54.904781 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 330/672] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 15 07:20:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 07:20:54.905932 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5e847a-2b3c-4b26-a84c-db296ae84c80 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:54.910231 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 343/673] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 15 07:20:54 2022] GET /identity/v3/role_assignments?user.id=3ecf6642b7b84d3c9756bb41ab3349d3&scope.project.id=4e9987720930473da840579654311bc2&role.id=0e661b5a67664692b933e5da9fc2e488 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:54.912181 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b853f52c-8066-4788-9b5f-09465b611b41 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:55.052169 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5e847a-2b3c-4b26-a84c-db296ae84c80 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:55.069611 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b853f52c-8066-4788-9b5f-09465b611b41 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:55.097288 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e46107-9c4c-40be-ae0b-b28df5da5485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:55.098133 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e46107-9c4c-40be-ae0b-b28df5da5485 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:55.100396 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e46107-9c4c-40be-ae0b-b28df5da5485 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:55.100396 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e46107-9c4c-40be-ae0b-b28df5da5485 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:55.114133 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e46107-9c4c-40be-ae0b-b28df5da5485 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:55.115598 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e46107-9c4c-40be-ae0b-b28df5da5485 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:55.116475 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed06e29e-ab26-485f-bccf-f3a40b29e4d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:55.117101 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed06e29e-ab26-485f-bccf-f3a40b29e4d9 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:55.117529 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed06e29e-ab26-485f-bccf-f3a40b29e4d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:55.117829 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed06e29e-ab26-485f-bccf-f3a40b29e4d9 None admin] PATH_INFO: `/v3/projects` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:55.118832 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed06e29e-ab26-485f-bccf-f3a40b29e4d9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:55.119137 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed06e29e-ab26-485f-bccf-f3a40b29e4d9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:55.120543 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed06e29e-ab26-485f-bccf-f3a40b29e4d9 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:55.134664 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-02e46107-9c4c-40be-ae0b-b28df5da5485 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:20:55.134664 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:55.134664 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:55.134664 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:55.134664 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:55.134664 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:55.134664 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:55.134664 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:55.134664 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:55.134664 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:55.134664 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:55.134664 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:55.134664 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 07:20:55.134664 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 07:20:55.134664 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 07:20:55.134664 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 07:20:55.135984 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:55.135984 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:55.135984 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:55.135984 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:55.135984 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:55.135984 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:55.135984 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:55.135984 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:55.135984 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:55.135984 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:55.135984 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:55.135984 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:55.135984 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:55.135984 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:55.135984 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:55.135984 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:55.135984 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 07:20:55.137000 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 07:20:55.137000 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:20:55.137000 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:20:55.137000 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 331/674] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:20:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:55.137000 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e46107-9c4c-40be-ae0b-b28df5da5485 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:55.143972 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 344/675] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 15 07:20:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:55.144820 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed06e29e-ab26-485f-bccf-f3a40b29e4d9 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:55.187475 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:55.209492 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:55.264677 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed06e29e-ab26-485f-bccf-f3a40b29e4d9 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:55.270587 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e46107-9c4c-40be-ae0b-b28df5da5485 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:55.303841 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7393ba-c54c-457e-a7c9-851194806d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:55.304603 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7393ba-c54c-457e-a7c9-851194806d3d None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:55.305074 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7393ba-c54c-457e-a7c9-851194806d3d None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:55.305450 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7393ba-c54c-457e-a7c9-851194806d3d None admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:55.306328 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7393ba-c54c-457e-a7c9-851194806d3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:55.306736 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7393ba-c54c-457e-a7c9-851194806d3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:55.308226 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7393ba-c54c-457e-a7c9-851194806d3d None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:55.311767 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8433fcbf-8942-4b4e-8ae9-a8bb27519d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:55.312590 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8433fcbf-8942-4b4e-8ae9-a8bb27519d88 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:55.313067 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8433fcbf-8942-4b4e-8ae9-a8bb27519d88 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:55.313474 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8433fcbf-8942-4b4e-8ae9-a8bb27519d88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:55.322124 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 345/676] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 15 07:20:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:55.323098 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7393ba-c54c-457e-a7c9-851194806d3d None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:55.325674 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8433fcbf-8942-4b4e-8ae9-a8bb27519d88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:55.329214 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8433fcbf-8942-4b4e-8ae9-a8bb27519d88 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:55.347068 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-8433fcbf-8942-4b4e-8ae9-a8bb27519d88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:20:55.347068 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:55.347068 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:55.347068 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:55.347068 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:55.347068 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:55.347068 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:55.347068 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:55.347068 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:55.347068 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:55.347068 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:55.347068 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:55.347068 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 07:20:55.347068 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 07:20:55.347068 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 07:20:55.347068 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 07:20:55.348256 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:55.348256 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:55.348256 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:55.348256 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:55.348256 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:55.348256 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:55.348256 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:55.348256 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:55.348256 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:55.348256 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:55.348256 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:55.348256 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:55.348256 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:55.348256 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:55.348256 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:55.348256 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:55.348256 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 07:20:55.349333 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 07:20:55.349333 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:20:55.349333 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:20:55.351906 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 332/677] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:20:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:55.352918 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8433fcbf-8942-4b4e-8ae9-a8bb27519d88 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:55.439405 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7393ba-c54c-457e-a7c9-851194806d3d None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:55.449813 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:55.462682 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:55.480014 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d67463-547c-4cd7-addd-49d660dae52e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:55.480751 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d67463-547c-4cd7-addd-49d660dae52e None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:55.481128 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d67463-547c-4cd7-addd-49d660dae52e None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:55.481761 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d67463-547c-4cd7-addd-49d660dae52e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:55.482494 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d67463-547c-4cd7-addd-49d660dae52e None admin] RBAC: Adding query filter params (role.id=0e661b5a67664692b933e5da9fc2e488, scope.project.id=4e9987720930473da840579654311bc2, user.id=a080c7ab9b684abca6828067f2572c3b) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:55.482611 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8433fcbf-8942-4b4e-8ae9-a8bb27519d88 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:55.482855 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d67463-547c-4cd7-addd-49d660dae52e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:55.484985 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d67463-547c-4cd7-addd-49d660dae52e None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:55.502161 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 346/678] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 15 07:20:55 2022] GET /identity/v3/role_assignments?user.id=a080c7ab9b684abca6828067f2572c3b&scope.project.id=4e9987720930473da840579654311bc2&role.id=0e661b5a67664692b933e5da9fc2e488 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:55.509735 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d67463-547c-4cd7-addd-49d660dae52e None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:55.534739 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2ef0f9-4f1c-4785-92d3-8e86690d3a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:55.535468 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2ef0f9-4f1c-4785-92d3-8e86690d3a09 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:55.535753 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2ef0f9-4f1c-4785-92d3-8e86690d3a09 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:55.536029 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2ef0f9-4f1c-4785-92d3-8e86690d3a09 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:55.545527 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2ef0f9-4f1c-4785-92d3-8e86690d3a09 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:55.547275 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2ef0f9-4f1c-4785-92d3-8e86690d3a09 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:55.562447 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-0f2ef0f9-4f1c-4785-92d3-8e86690d3a09 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 15 07:20:55.562447 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:55.562447 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:55.562447 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:55.562447 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:55.562447 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:55.562447 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:55.562447 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:55.562447 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:55.562447 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:55.562447 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:55.562447 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:55.562447 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 07:20:55.562447 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 07:20:55.562447 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 07:20:55.562447 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 07:20:55.563645 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:55.563645 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:55.563645 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 07:20:55.563645 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 07:20:55.563645 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 07:20:55.563645 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 07:20:55.563645 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:55.563645 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:55.563645 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:55.563645 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:55.563645 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:55.563645 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:55.563645 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:55.563645 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:55.563645 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:55.563645 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:55.563645 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:55.563645 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:55.564797 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:55.564797 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:55.564797 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 07:20:55.564797 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 07:20:55.564797 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 07:20:55.564797 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 07:20:55.564797 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 07:20:55.564797 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 07:20:55.564797 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 15 07:20:55.564797 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:20:55.564797 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 333/679] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 15 07:20:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:55.573382 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2ef0f9-4f1c-4785-92d3-8e86690d3a09 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:55.652372 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d67463-547c-4cd7-addd-49d660dae52e None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:55.696554 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07a01e6-c03c-4b4f-9054-0593e03541cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:55.697559 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07a01e6-c03c-4b4f-9054-0593e03541cc None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:55.698215 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07a01e6-c03c-4b4f-9054-0593e03541cc None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:55.698699 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07a01e6-c03c-4b4f-9054-0593e03541cc None admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:55.699986 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07a01e6-c03c-4b4f-9054-0593e03541cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:55.700715 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07a01e6-c03c-4b4f-9054-0593e03541cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:55.702728 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07a01e6-c03c-4b4f-9054-0593e03541cc None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:55.725650 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2ef0f9-4f1c-4785-92d3-8e86690d3a09 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:55.726227 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 347/680] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 15 07:20:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:55.727840 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365996cb-9af2-48d8-9278-0cb858df2b3b None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:55.728288 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365996cb-9af2-48d8-9278-0cb858df2b3b None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:55.728707 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365996cb-9af2-48d8-9278-0cb858df2b3b None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:55.730567 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 348/681] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:20:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:20:55.731514 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365996cb-9af2-48d8-9278-0cb858df2b3b None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:55.766989 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adc4f83-4a70-4870-b430-7a5143575fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:55.767672 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adc4f83-4a70-4870-b430-7a5143575fb4 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:55.768073 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adc4f83-4a70-4870-b430-7a5143575fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:55.768448 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adc4f83-4a70-4870-b430-7a5143575fb4 None admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:55.769674 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adc4f83-4a70-4870-b430-7a5143575fb4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:55.769889 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adc4f83-4a70-4870-b430-7a5143575fb4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:55.771577 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adc4f83-4a70-4870-b430-7a5143575fb4 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:55.777962 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:55.793806 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 334/682] 10.222.0.42 () {56 vars in 1135 bytes} [Thu Sep 15 07:20:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:55.795281 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47777ff-1978-4f68-b833-8f84f046f5f2 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:55.795668 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47777ff-1978-4f68-b833-8f84f046f5f2 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:55.796031 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47777ff-1978-4f68-b833-8f84f046f5f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:55.804531 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:55.817155 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-e47777ff-1978-4f68-b833-8f84f046f5f2 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:20:55.818921 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-e47777ff-1978-4f68-b833-8f84f046f5f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:20:55.878070 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365996cb-9af2-48d8-9278-0cb858df2b3b None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:55.921180 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 335/683] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:20:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 07:20:55.922665 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47777ff-1978-4f68-b833-8f84f046f5f2 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:55.923073 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab89e19b-96f9-4e3c-9db6-88e2ed9227ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:55.923888 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab89e19b-96f9-4e3c-9db6-88e2ed9227ce None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:55.924177 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab89e19b-96f9-4e3c-9db6-88e2ed9227ce None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:55.924453 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab89e19b-96f9-4e3c-9db6-88e2ed9227ce None admin] PATH_INFO: `/v3/projects/service` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:55.934056 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab89e19b-96f9-4e3c-9db6-88e2ed9227ce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:55.936160 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab89e19b-96f9-4e3c-9db6-88e2ed9227ce None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:55.963289 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-ab89e19b-96f9-4e3c-9db6-88e2ed9227ce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 07:20:55.963289 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:55.963289 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:55.963289 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:55.963289 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:55.963289 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:55.963289 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:55.963289 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:55.963289 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:55.963289 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:55.963289 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:55.963289 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:55.963289 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 07:20:55.963289 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 07:20:55.963289 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 07:20:55.964224 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 07:20:55.964224 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:55.964224 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:55.964224 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:55.964224 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:55.964224 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:55.964224 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:55.964224 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:55.964224 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:55.964224 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:55.964224 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:55.964224 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:55.964224 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:55.964224 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:55.964224 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:55.964224 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:55.964224 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:55.965127 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 07:20:55.965127 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 07:20:55.965127 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 07:20:55.965127 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 07:20:55.965127 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 07:20:55.965127 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 07:20:55.965127 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 07:20:55.965127 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:20:55.967063 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 349/684] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 15 07:20:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:55.972280 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab89e19b-96f9-4e3c-9db6-88e2ed9227ce None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:56.089160 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47777ff-1978-4f68-b833-8f84f046f5f2 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:56.127437 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4790dd2e-52ec-4f76-9840-0ec0d68a9431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:56.128058 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4790dd2e-52ec-4f76-9840-0ec0d68a9431 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:56.128339 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4790dd2e-52ec-4f76-9840-0ec0d68a9431 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:56.128614 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4790dd2e-52ec-4f76-9840-0ec0d68a9431 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:56.131491 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:56.132314 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:56.145612 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab89e19b-96f9-4e3c-9db6-88e2ed9227ce None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:56.181439 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cb0ba6-b96f-40e3-bc44-e82646509011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:56.182089 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cb0ba6-b96f-40e3-bc44-e82646509011 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:56.182426 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cb0ba6-b96f-40e3-bc44-e82646509011 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:56.182705 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cb0ba6-b96f-40e3-bc44-e82646509011 None admin] PATH_INFO: `/v3/projects` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:56.183437 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cb0ba6-b96f-40e3-bc44-e82646509011 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:56.183755 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cb0ba6-b96f-40e3-bc44-e82646509011 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:56.185886 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cb0ba6-b96f-40e3-bc44-e82646509011 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:56.205771 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4790dd2e-52ec-4f76-9840-0ec0d68a9431 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:56.207161 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4790dd2e-52ec-4f76-9840-0ec0d68a9431 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:56.221117 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 350/685] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 15 07:20:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:56.224644 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cb0ba6-b96f-40e3-bc44-e82646509011 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:56.337951 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cb0ba6-b96f-40e3-bc44-e82646509011 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:56.341119 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 336/686] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 15 07:20:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 07:20:56.342955 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4790dd2e-52ec-4f76-9840-0ec0d68a9431 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:56.379050 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d849195c-4cd3-4b0b-9cf1-faeded9ac2a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:56.379801 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d849195c-4cd3-4b0b-9cf1-faeded9ac2a6 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:56.380092 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d849195c-4cd3-4b0b-9cf1-faeded9ac2a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:56.380372 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d849195c-4cd3-4b0b-9cf1-faeded9ac2a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:56.382688 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:56.392775 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d849195c-4cd3-4b0b-9cf1-faeded9ac2a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:56.394489 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d849195c-4cd3-4b0b-9cf1-faeded9ac2a6 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:56.412411 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-d849195c-4cd3-4b0b-9cf1-faeded9ac2a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:20:56.412411 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:56.412411 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:56.412411 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:56.412411 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:56.412411 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:56.412411 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:56.412411 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:56.412411 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:56.412411 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:56.412411 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:56.412411 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:56.412411 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 07:20:56.412411 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 07:20:56.412411 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 07:20:56.412411 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 07:20:56.413635 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:56.413635 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:56.413635 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:56.413635 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:56.413635 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:56.413635 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:56.413635 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:56.413635 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:56.413635 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:56.413635 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:56.413635 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:56.413635 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:56.413635 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:56.413635 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:56.413635 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:56.413635 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:56.413635 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 07:20:56.414603 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 07:20:56.414603 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:20:56.414603 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:20:56.416300 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 351/687] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:20:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:56.417396 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d849195c-4cd3-4b0b-9cf1-faeded9ac2a6 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:56.422744 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:56.453022 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4790dd2e-52ec-4f76-9840-0ec0d68a9431 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:56.489317 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6807b89b-9c81-4f3c-877d-554792fb7829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:56.489862 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6807b89b-9c81-4f3c-877d-554792fb7829 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:56.490135 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6807b89b-9c81-4f3c-877d-554792fb7829 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:56.490419 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6807b89b-9c81-4f3c-877d-554792fb7829 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:56.491134 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6807b89b-9c81-4f3c-877d-554792fb7829 None admin] RBAC: Adding query filter params (role.id=0e661b5a67664692b933e5da9fc2e488, scope.project.id=4e9987720930473da840579654311bc2, user.id=5315996432f44bffa3460b2744c10f0a) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:56.491442 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6807b89b-9c81-4f3c-877d-554792fb7829 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:56.492853 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6807b89b-9c81-4f3c-877d-554792fb7829 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:56.504886 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 337/688] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 15 07:20:56 2022] GET /identity/v3/role_assignments?user.id=5315996432f44bffa3460b2744c10f0a&scope.project.id=4e9987720930473da840579654311bc2&role.id=0e661b5a67664692b933e5da9fc2e488 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:56.505895 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6807b89b-9c81-4f3c-877d-554792fb7829 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:56.529261 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d849195c-4cd3-4b0b-9cf1-faeded9ac2a6 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:56.569049 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fc7ea8-7252-45da-bb24-f0fa8a2b1652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:56.570133 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fc7ea8-7252-45da-bb24-f0fa8a2b1652 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:56.570620 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fc7ea8-7252-45da-bb24-f0fa8a2b1652 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:56.571092 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fc7ea8-7252-45da-bb24-f0fa8a2b1652 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:56.584075 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fc7ea8-7252-45da-bb24-f0fa8a2b1652 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:56.585907 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fc7ea8-7252-45da-bb24-f0fa8a2b1652 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:56.602161 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-51fc7ea8-7252-45da-bb24-f0fa8a2b1652 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:20:56.602161 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:56.602161 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:56.602161 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:56.602161 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:56.602161 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:56.602161 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:56.602161 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:56.602161 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:56.602161 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:56.602161 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:56.602161 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:56.602161 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 07:20:56.602161 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 07:20:56.602161 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 07:20:56.602161 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 07:20:56.603171 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:56.603171 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:56.603171 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:56.603171 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:56.603171 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:56.603171 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:56.603171 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:56.603171 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:56.603171 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:56.603171 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:56.603171 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:56.603171 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:56.603171 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:56.603171 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:56.603171 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:56.603171 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:56.603171 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 07:20:56.604078 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 07:20:56.604078 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:20:56.604078 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:20:56.611886 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 352/689] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:20:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:56.613503 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fc7ea8-7252-45da-bb24-f0fa8a2b1652 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:56.709774 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:56.725121 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:56.729303 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6807b89b-9c81-4f3c-877d-554792fb7829 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:56.764831 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fc7ea8-7252-45da-bb24-f0fa8a2b1652 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:56.805497 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd40269-6b26-4154-8827-de631dae933a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:56.806334 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7272678a-24fa-4bf7-a294-d2dd792d40ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:56.806789 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd40269-6b26-4154-8827-de631dae933a None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:56.806969 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7272678a-24fa-4bf7-a294-d2dd792d40ea None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:56.807233 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7272678a-24fa-4bf7-a294-d2dd792d40ea None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:56.807503 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7272678a-24fa-4bf7-a294-d2dd792d40ea None admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:56.807843 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd40269-6b26-4154-8827-de631dae933a None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:56.808160 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7272678a-24fa-4bf7-a294-d2dd792d40ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:56.808461 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7272678a-24fa-4bf7-a294-d2dd792d40ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:56.808729 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd40269-6b26-4154-8827-de631dae933a None admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:56.809558 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd40269-6b26-4154-8827-de631dae933a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:56.810260 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7272678a-24fa-4bf7-a294-d2dd792d40ea None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:56.810583 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd40269-6b26-4154-8827-de631dae933a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:56.812844 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd40269-6b26-4154-8827-de631dae933a None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:56.831428 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 353/690] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 15 07:20:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:56.832335 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 338/691] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 15 07:20:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:56.834633 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd40269-6b26-4154-8827-de631dae933a None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:56.837325 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7272678a-24fa-4bf7-a294-d2dd792d40ea None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:56.950013 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd40269-6b26-4154-8827-de631dae933a None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:56.952664 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7272678a-24fa-4bf7-a294-d2dd792d40ea None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:56.974714 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:56.989701 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55694b8-2e34-4352-afa1-a681d1036538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:56.990658 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55694b8-2e34-4352-afa1-a681d1036538 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:56.991144 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55694b8-2e34-4352-afa1-a681d1036538 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:56.991714 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55694b8-2e34-4352-afa1-a681d1036538 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:56.994375 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87ef4c4-0cfc-43fd-abe4-78810002a672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:56.995299 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87ef4c4-0cfc-43fd-abe4-78810002a672 None admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:56.995410 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87ef4c4-0cfc-43fd-abe4-78810002a672 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:56.995713 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87ef4c4-0cfc-43fd-abe4-78810002a672 None admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:56.996422 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87ef4c4-0cfc-43fd-abe4-78810002a672 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:56.998612 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87ef4c4-0cfc-43fd-abe4-78810002a672 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:57.001676 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55694b8-2e34-4352-afa1-a681d1036538 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:57.003353 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55694b8-2e34-4352-afa1-a681d1036538 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:57.016831 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-d55694b8-2e34-4352-afa1-a681d1036538 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 07:20:57.016831 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:57.016831 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:57.016831 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:57.016831 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:57.016831 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:57.016831 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:57.016831 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:57.016831 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:57.016831 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:57.016831 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:57.016831 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:57.016831 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 07:20:57.016831 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 07:20:57.016831 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 07:20:57.017851 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 07:20:57.017851 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:57.017851 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:57.017851 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:57.017851 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:57.017851 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:57.017851 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:57.017851 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:57.017851 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:57.017851 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:57.017851 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:57.017851 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:57.017851 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:57.017851 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:57.017851 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:57.017851 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:57.017851 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:57.018719 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 07:20:57.018719 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 07:20:57.018719 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 07:20:57.018719 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 07:20:57.018719 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 07:20:57.018719 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 07:20:57.018719 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 07:20:57.018719 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:20:57.021737 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 354/692] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 15 07:20:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:57.023834 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55694b8-2e34-4352-afa1-a681d1036538 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:57.104596 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 339/693] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Sep 15 07:20:56 2022] POST /identity/v3/users => generated 254 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 07:20:57.143454 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55694b8-2e34-4352-afa1-a681d1036538 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:57.190231 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b7b925-1139-4616-9636-ee97220b5c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:57.191253 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b7b925-1139-4616-9636-ee97220b5c44 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:57.191743 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b7b925-1139-4616-9636-ee97220b5c44 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:57.192221 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b7b925-1139-4616-9636-ee97220b5c44 None admin] PATH_INFO: `/v3/projects` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:57.193168 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b7b925-1139-4616-9636-ee97220b5c44 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:57.193734 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b7b925-1139-4616-9636-ee97220b5c44 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:57.195438 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b7b925-1139-4616-9636-ee97220b5c44 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:57.210628 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 355/694] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 15 07:20:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:20:57.216511 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87ef4c4-0cfc-43fd-abe4-78810002a672 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:57.247009 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:57.366595 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87ef4c4-0cfc-43fd-abe4-78810002a672 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:57.404062 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6714734c-e00b-408f-9a6c-a30619ef8fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:57.404681 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6714734c-e00b-408f-9a6c-a30619ef8fab None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:57.404963 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6714734c-e00b-408f-9a6c-a30619ef8fab None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:57.405241 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6714734c-e00b-408f-9a6c-a30619ef8fab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:57.406196 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6714734c-e00b-408f-9a6c-a30619ef8fab None admin] RBAC: Adding query filter params (role.id=0e661b5a67664692b933e5da9fc2e488, scope.project.id=4e9987720930473da840579654311bc2, user.id=72b90d35087e4d7a80e9417082a1541f) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:57.406516 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6714734c-e00b-408f-9a6c-a30619ef8fab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:57.407942 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6714734c-e00b-408f-9a6c-a30619ef8fab None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:57.414770 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 340/695] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 15 07:20:57 2022] GET /identity/v3/role_assignments?user.id=72b90d35087e4d7a80e9417082a1541f&scope.project.id=4e9987720930473da840579654311bc2&role.id=0e661b5a67664692b933e5da9fc2e488 => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:57.449780 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71b3e43-047f-4e22-822c-b30f68784acb None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:57.450110 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71b3e43-047f-4e22-822c-b30f68784acb None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:57.450369 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71b3e43-047f-4e22-822c-b30f68784acb None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:57.451733 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 356/696] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:20:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:20:57.458216 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212f32a9-93b8-4f70-8982-9e87b34098f1 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:57.458523 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212f32a9-93b8-4f70-8982-9e87b34098f1 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:57.458783 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212f32a9-93b8-4f70-8982-9e87b34098f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:57.481513 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-212f32a9-93b8-4f70-8982-9e87b34098f1 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:20:57.483260 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-212f32a9-93b8-4f70-8982-9e87b34098f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:20:57.564818 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 341/697] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:20:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 07:20:57.851110 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71b3e43-047f-4e22-822c-b30f68784acb None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:57.900379 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:57.979550 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71b3e43-047f-4e22-822c-b30f68784acb None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:58.016249 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e72c75-9fe7-4cc2-b870-0c0e122d9368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:58.016891 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e72c75-9fe7-4cc2-b870-0c0e122d9368 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:58.017171 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e72c75-9fe7-4cc2-b870-0c0e122d9368 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:58.017444 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e72c75-9fe7-4cc2-b870-0c0e122d9368 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:58.018070 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e72c75-9fe7-4cc2-b870-0c0e122d9368 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:58.019500 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e72c75-9fe7-4cc2-b870-0c0e122d9368 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:58.030520 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-68e72c75-9fe7-4cc2-b870-0c0e122d9368 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 15 07:20:58.030520 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:58.030520 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 07:20:58.030520 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 07:20:58.030520 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 07:20:58.030520 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 07:20:58.030520 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 07:20:58.030520 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 07:20:58.030520 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 15 07:20:58.030520 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:20:58.030520 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 07:20:58.030520 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:20:58.030520 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:58.030520 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:58.030520 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:58.030520 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:58.030520 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:58.032376 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:58.032376 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:58.032376 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:58.032376 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:58.032376 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:58.032376 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:58.032376 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 07:20:58.032376 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 07:20:58.032376 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 07:20:58.032376 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 07:20:58.032376 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:58.032376 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:58.032376 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:58.032376 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:58.032376 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:58.032376 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:58.033213 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:58.033213 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:58.033213 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:58.033213 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:58.033213 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:58.033213 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:58.033213 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:58.033213 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:58.033213 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:58.033213 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:58.033213 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 07:20:58.033213 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 07:20:58.033213 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 15 07:20:58.033213 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:20:58.033213 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 357/698] 10.222.0.42 () {56 vars in 1129 bytes} [Thu Sep 15 07:20:57 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:58.038537 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212f32a9-93b8-4f70-8982-9e87b34098f1 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:58.053857 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:58.083710 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f019682a-01c5-4887-838f-e827d5748359 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:58.084024 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f019682a-01c5-4887-838f-e827d5748359 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:58.084255 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f019682a-01c5-4887-838f-e827d5748359 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:58.085857 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 358/699] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:20:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:20:58.091095 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616a18be-dbd0-4733-a212-495a04fe44cf None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:58.091381 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616a18be-dbd0-4733-a212-495a04fe44cf None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:58.091659 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616a18be-dbd0-4733-a212-495a04fe44cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:58.130177 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-616a18be-dbd0-4733-a212-495a04fe44cf None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:20:58.131319 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-616a18be-dbd0-4733-a212-495a04fe44cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:20:58.205282 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 359/700] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:20:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 07:20:58.255730 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212f32a9-93b8-4f70-8982-9e87b34098f1 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:58.315210 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf24259-933d-47f3-9066-997b17d8681b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:58.315833 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf24259-933d-47f3-9066-997b17d8681b None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:58.316114 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf24259-933d-47f3-9066-997b17d8681b None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:58.316390 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf24259-933d-47f3-9066-997b17d8681b None admin] PATH_INFO: `/v3/services` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:58.317029 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf24259-933d-47f3-9066-997b17d8681b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:58.317331 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf24259-933d-47f3-9066-997b17d8681b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:58.318720 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf24259-933d-47f3-9066-997b17d8681b None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:58.330483 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 342/701] 10.222.0.42 () {56 vars in 1124 bytes} [Thu Sep 15 07:20:58 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:58.336022 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616a18be-dbd0-4733-a212-495a04fe44cf None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:58.379686 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:58.456759 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616a18be-dbd0-4733-a212-495a04fe44cf None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:58.499814 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8b8246-dfcd-4519-9939-8bb6df789181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:58.500428 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8b8246-dfcd-4519-9939-8bb6df789181 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:58.500709 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8b8246-dfcd-4519-9939-8bb6df789181 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:58.500985 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8b8246-dfcd-4519-9939-8bb6df789181 None admin] PATH_INFO: `/v3/services` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:58.501640 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8b8246-dfcd-4519-9939-8bb6df789181 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:58.501952 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8b8246-dfcd-4519-9939-8bb6df789181 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:58.503387 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8b8246-dfcd-4519-9939-8bb6df789181 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:58.509459 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 360/702] 10.222.0.42 () {56 vars in 1124 bytes} [Thu Sep 15 07:20:58 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:58.579055 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf24259-933d-47f3-9066-997b17d8681b None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:58.688201 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:58.729527 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf24259-933d-47f3-9066-997b17d8681b None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:58.778842 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2838e486-88d7-4abc-8c98-85fa68c11049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:58.779895 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2838e486-88d7-4abc-8c98-85fa68c11049 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:58.780389 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2838e486-88d7-4abc-8c98-85fa68c11049 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:58.781385 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2838e486-88d7-4abc-8c98-85fa68c11049 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:58.793683 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2838e486-88d7-4abc-8c98-85fa68c11049 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:20:58.799252 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-2838e486-88d7-4abc-8c98-85fa68c11049 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 07:20:58.799252 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:58.799252 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:58.799252 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:58.799252 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:58.799252 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:58.799252 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:58.799252 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:58.799252 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:58.799252 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:58.799252 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:58.799252 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:58.799252 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:20:58.799252 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:20:58.799252 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:20:58.799252 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:20:58.800189 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:20:58.800189 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:20:58.800189 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:20:58.800189 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:20:58.800189 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:58.800189 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:58.800189 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:58.800189 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:58.800189 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:58.800189 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:58.800189 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:58.800189 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:58.800189 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:58.800189 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:58.800189 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:58.800189 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:58.801055 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:58.801055 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:58.801055 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:58.801055 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:58.801055 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:20:58.801055 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:20:58.801055 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:20:58.801055 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:20:58.801055 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:20:58.801055 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:20:58.801055 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 07:20:58.801055 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:20:58.801055 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 343/703] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 15 07:20:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:58.806133 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8b8246-dfcd-4519-9939-8bb6df789181 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:58.907746 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:58.920711 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8b8246-dfcd-4519-9939-8bb6df789181 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:58.974381 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4343d4f-d82d-4ed4-bb39-ed7bc3b1963f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:58.975003 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4343d4f-d82d-4ed4-bb39-ed7bc3b1963f None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:58.975281 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4343d4f-d82d-4ed4-bb39-ed7bc3b1963f None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:58.975552 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4343d4f-d82d-4ed4-bb39-ed7bc3b1963f None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:58.976213 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4343d4f-d82d-4ed4-bb39-ed7bc3b1963f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:58.976516 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4343d4f-d82d-4ed4-bb39-ed7bc3b1963f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:58.978038 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4343d4f-d82d-4ed4-bb39-ed7bc3b1963f None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:58.993846 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 361/704] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 15 07:20:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:59.002676 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2838e486-88d7-4abc-8c98-85fa68c11049 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:59.155481 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2838e486-88d7-4abc-8c98-85fa68c11049 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:59.198141 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:59.204266 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7506f9-dfd6-43ee-bd39-e91038978bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:59.204948 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7506f9-dfd6-43ee-bd39-e91038978bfa None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:59.205247 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7506f9-dfd6-43ee-bd39-e91038978bfa None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:59.205527 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7506f9-dfd6-43ee-bd39-e91038978bfa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:59.215676 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7506f9-dfd6-43ee-bd39-e91038978bfa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:59.217114 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7506f9-dfd6-43ee-bd39-e91038978bfa None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:59.233783 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-7e7506f9-dfd6-43ee-bd39-e91038978bfa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:20:59.233783 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:59.233783 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:59.233783 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:59.233783 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:59.233783 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:59.233783 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:59.233783 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:59.233783 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:59.233783 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:59.233783 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:59.233783 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:59.233783 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 07:20:59.233783 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 07:20:59.233783 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 07:20:59.233783 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 07:20:59.234817 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:59.234817 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:59.234817 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:59.234817 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:59.234817 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:59.234817 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:59.234817 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:59.234817 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:59.234817 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:59.234817 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:59.234817 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:59.234817 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:59.234817 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:59.234817 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:59.234817 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:59.234817 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:59.234817 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 07:20:59.235747 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 07:20:59.235747 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:20:59.235747 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:20:59.235931 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 344/705] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:20:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:59.241062 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4343d4f-d82d-4ed4-bb39-ed7bc3b1963f None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:59.387868 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4343d4f-d82d-4ed4-bb39-ed7bc3b1963f None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:59.435689 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00ede18-fa58-4e26-b11f-569da2bcfd15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:59.436705 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00ede18-fa58-4e26-b11f-569da2bcfd15 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:59.437853 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00ede18-fa58-4e26-b11f-569da2bcfd15 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:59.438367 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00ede18-fa58-4e26-b11f-569da2bcfd15 None admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:59.439220 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00ede18-fa58-4e26-b11f-569da2bcfd15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:59.439728 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00ede18-fa58-4e26-b11f-569da2bcfd15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:59.441334 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00ede18-fa58-4e26-b11f-569da2bcfd15 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:59.459441 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 362/706] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 15 07:20:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:59.464507 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7506f9-dfd6-43ee-bd39-e91038978bfa None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:59.619466 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7506f9-dfd6-43ee-bd39-e91038978bfa None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:59.666481 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d783b2-21bc-4e3d-a91f-becf4021e95d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:59.667525 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d783b2-21bc-4e3d-a91f-becf4021e95d None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:59.668017 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d783b2-21bc-4e3d-a91f-becf4021e95d None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:59.668491 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d783b2-21bc-4e3d-a91f-becf4021e95d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:59.681115 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d783b2-21bc-4e3d-a91f-becf4021e95d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:59.682956 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d783b2-21bc-4e3d-a91f-becf4021e95d None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:59.702608 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-a6d783b2-21bc-4e3d-a91f-becf4021e95d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 15 07:20:59.702608 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:20:59.702608 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:20:59.702608 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:20:59.702608 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:20:59.702608 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:20:59.702608 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:20:59.702608 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:20:59.702608 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:20:59.702608 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:20:59.702608 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:20:59.702608 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:20:59.702608 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 07:20:59.702608 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 07:20:59.702608 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 07:20:59.702608 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 07:20:59.704595 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:20:59.704595 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:20:59.704595 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 07:20:59.704595 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 07:20:59.704595 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 07:20:59.704595 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 07:20:59.704595 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:20:59.704595 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:20:59.704595 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:20:59.704595 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:20:59.704595 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:20:59.704595 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:20:59.704595 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:20:59.704595 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:20:59.704595 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:20:59.704595 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:20:59.704595 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:20:59.704595 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:20:59.705543 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:20:59.705543 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:20:59.705543 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 07:20:59.705543 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 07:20:59.705543 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 07:20:59.705543 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 07:20:59.705543 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 07:20:59.705543 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 07:20:59.705543 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 15 07:20:59.705543 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:20:59.708314 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 345/707] 10.222.0.42 () {56 vars in 1093 bytes} [Thu Sep 15 07:20:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:20:59.713946 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00ede18-fa58-4e26-b11f-569da2bcfd15 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:20:59.743586 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:59.817170 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143d0159-5f9f-463c-897a-1f9f7e0a67d6 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:59.817520 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143d0159-5f9f-463c-897a-1f9f7e0a67d6 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:59.817785 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143d0159-5f9f-463c-897a-1f9f7e0a67d6 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:59.820556 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 346/708] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:20:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 07:20:59.830342 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cc7591-4583-4b4d-91e2-53ea38d128a3 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:59.830623 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cc7591-4583-4b4d-91e2-53ea38d128a3 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:59.830898 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cc7591-4583-4b4d-91e2-53ea38d128a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:59.855777 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-72cc7591-4583-4b4d-91e2-53ea38d128a3 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:20:59.858482 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-72cc7591-4583-4b4d-91e2-53ea38d128a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:20:59.874795 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00ede18-fa58-4e26-b11f-569da2bcfd15 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:20:59.952986 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:20:59.962339 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7570bf18-720a-4240-b3d7-c113b12641e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:20:59.963044 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7570bf18-720a-4240-b3d7-c113b12641e0 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:20:59.963248 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7570bf18-720a-4240-b3d7-c113b12641e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:20:59.963516 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7570bf18-720a-4240-b3d7-c113b12641e0 None admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:20:59.969025 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7570bf18-720a-4240-b3d7-c113b12641e0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:20:59.969025 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7570bf18-720a-4240-b3d7-c113b12641e0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:20:59.969025 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7570bf18-720a-4240-b3d7-c113b12641e0 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:20:59.986456 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 347/709] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:20:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 07:20:59.987269 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 363/710] 10.222.0.42 () {56 vars in 1133 bytes} [Thu Sep 15 07:20:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:20:59.993635 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cc7591-4583-4b4d-91e2-53ea38d128a3 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:00.097192 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cc7591-4583-4b4d-91e2-53ea38d128a3 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:00.136116 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ec7c31-11ad-4d7b-8c99-f99d00ea2f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:00.136740 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ec7c31-11ad-4d7b-8c99-f99d00ea2f62 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:00.137015 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ec7c31-11ad-4d7b-8c99-f99d00ea2f62 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:00.137285 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ec7c31-11ad-4d7b-8c99-f99d00ea2f62 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:00.181955 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:00.208729 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ec7c31-11ad-4d7b-8c99-f99d00ea2f62 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:00.210366 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ec7c31-11ad-4d7b-8c99-f99d00ea2f62 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:00.227767 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad37c8-4db0-45f8-a5cb-9e934da06b2f None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:00.228263 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad37c8-4db0-45f8-a5cb-9e934da06b2f None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:00.228376 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad37c8-4db0-45f8-a5cb-9e934da06b2f None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:00.230737 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 364/711] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:21:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:21:00.236013 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bd4448-2fba-47e5-bd85-b9946b103e1d None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:00.236530 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bd4448-2fba-47e5-bd85-b9946b103e1d None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:00.237015 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bd4448-2fba-47e5-bd85-b9946b103e1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:00.269013 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-99bd4448-2fba-47e5-bd85-b9946b103e1d None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:21:00.269013 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-99bd4448-2fba-47e5-bd85-b9946b103e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:21:00.383721 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 365/712] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:21:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 07:21:00.384685 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bd4448-2fba-47e5-bd85-b9946b103e1d None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:00.387968 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:00.396167 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 348/713] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 15 07:20:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 15 07:21:00.405599 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ec7c31-11ad-4d7b-8c99-f99d00ea2f62 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:00.523630 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:00.541693 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bd4448-2fba-47e5-bd85-b9946b103e1d None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:00.552319 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ec7c31-11ad-4d7b-8c99-f99d00ea2f62 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:00.587176 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c8f3a3-bf17-4e1f-bddc-c6d6e4492e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:00.587925 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c8f3a3-bf17-4e1f-bddc-c6d6e4492e43 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:00.588680 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c8f3a3-bf17-4e1f-bddc-c6d6e4492e43 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:00.589041 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c8f3a3-bf17-4e1f-bddc-c6d6e4492e43 None admin] PATH_INFO: `/v3/services/network` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:00.590030 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c8f3a3-bf17-4e1f-bddc-c6d6e4492e43 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:00.591492 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c8f3a3-bf17-4e1f-bddc-c6d6e4492e43 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:00.593817 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faa640b-f610-42f5-9064-4c6bf39bc782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:00.594623 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faa640b-f610-42f5-9064-4c6bf39bc782 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:00.595224 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faa640b-f610-42f5-9064-4c6bf39bc782 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:00.595678 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faa640b-f610-42f5-9064-4c6bf39bc782 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:00.601609 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-61c8f3a3-bf17-4e1f-bddc-c6d6e4492e43 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 15 07:21:00.601609 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:00.601609 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 07:21:00.601609 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 07:21:00.601609 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 07:21:00.601609 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 07:21:00.601609 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 07:21:00.601609 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 07:21:00.601609 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 15 07:21:00.601609 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:21:00.601609 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 07:21:00.601609 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:21:00.601609 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:00.601609 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:00.601609 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:00.601609 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:00.601609 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:00.602806 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:00.602806 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:00.602806 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:00.602806 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:00.602806 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:00.602806 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:00.602806 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 07:21:00.602806 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 07:21:00.602806 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 07:21:00.602806 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 07:21:00.602806 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:00.602806 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:00.602806 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:00.602806 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:00.602806 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:00.602806 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:00.603824 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:00.603824 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:00.603824 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:00.603824 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:00.603824 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:00.603824 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:00.603824 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:00.603824 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:00.603824 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:00.603824 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:00.603824 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 07:21:00.603824 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 07:21:00.603824 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 15 07:21:00.603824 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:21:00.609730 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 366/714] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 15 07:21:00 2022] GET /identity/v3/services/network => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:21:00.612880 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8588bc34-ace7-47ab-8bd5-171b3ff9549a None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:00.613210 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8588bc34-ace7-47ab-8bd5-171b3ff9549a None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:00.613427 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8588bc34-ace7-47ab-8bd5-171b3ff9549a None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:00.616900 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 367/715] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:21:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:21:00.616900 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8588bc34-ace7-47ab-8bd5-171b3ff9549a None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:00.619876 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faa640b-f610-42f5-9064-4c6bf39bc782 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:00.621388 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faa640b-f610-42f5-9064-4c6bf39bc782 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:00.642365 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-1faa640b-f610-42f5-9064-4c6bf39bc782 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:21:00.642365 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:00.642365 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:00.642365 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:00.642365 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:00.642365 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:00.642365 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:00.642365 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:00.642365 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:00.642365 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:00.642365 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:00.642365 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:00.642365 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 07:21:00.642365 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 07:21:00.642365 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 07:21:00.642365 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 07:21:00.643413 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:00.643413 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:00.643413 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:00.643413 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:00.643413 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:00.643413 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:00.643413 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:00.643413 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:00.643413 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:00.643413 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:00.643413 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:00.643413 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:00.643413 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:00.643413 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:00.643413 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:00.643413 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:00.643413 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 07:21:00.644432 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 07:21:00.644432 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:21:00.644432 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:21:00.646941 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 349/716] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:21:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:21:00.648479 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7fd33d-29e8-42ec-a998-9db062de2d89 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:00.648902 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7fd33d-29e8-42ec-a998-9db062de2d89 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:00.649311 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7fd33d-29e8-42ec-a998-9db062de2d89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:00.669377 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-cc7fd33d-29e8-42ec-a998-9db062de2d89 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:21:00.670873 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-cc7fd33d-29e8-42ec-a998-9db062de2d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:21:00.695098 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:00.735859 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8588bc34-ace7-47ab-8bd5-171b3ff9549a None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:00.747051 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 350/717] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:21:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 07:21:00.747741 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7fd33d-29e8-42ec-a998-9db062de2d89 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:00.785225 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaf5a07-8fd3-4d7f-a8f5-eff45819aaf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:00.786563 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaf5a07-8fd3-4d7f-a8f5-eff45819aaf3 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:00.787069 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaf5a07-8fd3-4d7f-a8f5-eff45819aaf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:00.787539 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaf5a07-8fd3-4d7f-a8f5-eff45819aaf3 None admin] PATH_INFO: `/v3/services` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:00.789504 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaf5a07-8fd3-4d7f-a8f5-eff45819aaf3 None admin] RBAC: Adding query filter params (name=network) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:00.790041 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaf5a07-8fd3-4d7f-a8f5-eff45819aaf3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:00.791673 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaf5a07-8fd3-4d7f-a8f5-eff45819aaf3 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:00.798253 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 368/718] 10.222.0.42 () {56 vars in 1112 bytes} [Thu Sep 15 07:21:00 2022] GET /identity/v3/services?name=network => generated 124 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:00.800070 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaf5a07-8fd3-4d7f-a8f5-eff45819aaf3 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:00.815473 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:00.870544 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7fd33d-29e8-42ec-a998-9db062de2d89 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:00.910189 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3da0561-55b1-4073-b2fe-92a6b345ed53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:00.911196 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3da0561-55b1-4073-b2fe-92a6b345ed53 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:00.911705 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3da0561-55b1-4073-b2fe-92a6b345ed53 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:00.912307 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3da0561-55b1-4073-b2fe-92a6b345ed53 None admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:00.913242 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3da0561-55b1-4073-b2fe-92a6b345ed53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:00.913761 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3da0561-55b1-4073-b2fe-92a6b345ed53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:00.915324 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3da0561-55b1-4073-b2fe-92a6b345ed53 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:00.928135 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 351/719] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 15 07:21:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:00.929284 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3da0561-55b1-4073-b2fe-92a6b345ed53 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:00.933438 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaf5a07-8fd3-4d7f-a8f5-eff45819aaf3 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:00.978181 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:00.983858 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572a9a61-bc4b-4d83-a0fe-cd5dde92d5f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:00.985059 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572a9a61-bc4b-4d83-a0fe-cd5dde92d5f8 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:00.985497 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572a9a61-bc4b-4d83-a0fe-cd5dde92d5f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:00.985949 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572a9a61-bc4b-4d83-a0fe-cd5dde92d5f8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:00.998303 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572a9a61-bc4b-4d83-a0fe-cd5dde92d5f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:21:01.004128 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-572a9a61-bc4b-4d83-a0fe-cd5dde92d5f8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 07:21:01.004128 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:01.004128 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:01.004128 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:01.004128 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:01.004128 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:01.004128 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:01.004128 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:01.004128 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:01.004128 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:01.004128 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:01.004128 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:01.004128 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:21:01.004128 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:21:01.004128 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:21:01.004128 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:21:01.005081 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:21:01.005081 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:21:01.005081 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:21:01.005081 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:21:01.005081 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:01.005081 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:01.005081 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:01.005081 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:01.005081 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:01.005081 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:01.005081 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:01.005081 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:01.005081 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:01.005081 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:01.005081 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:01.005081 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:01.005962 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:01.005962 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:01.005962 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:01.005962 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:01.005962 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:21:01.005962 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:21:01.005962 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:21:01.005962 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:21:01.005962 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:21:01.005962 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:21:01.005962 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 07:21:01.005962 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:21:01.011897 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 369/720] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Sep 15 07:21:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:21:01.011897 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572a9a61-bc4b-4d83-a0fe-cd5dde92d5f8 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:01.044235 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3da0561-55b1-4073-b2fe-92a6b345ed53 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:01.062586 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:01.084007 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0abefcc-3aa9-4763-b852-f2f9bdeb38bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:01.084850 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0abefcc-3aa9-4763-b852-f2f9bdeb38bf None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:01.085135 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0abefcc-3aa9-4763-b852-f2f9bdeb38bf None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:01.085435 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0abefcc-3aa9-4763-b852-f2f9bdeb38bf None admin] PATH_INFO: `/v3/services` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:01.086084 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0abefcc-3aa9-4763-b852-f2f9bdeb38bf None admin] RBAC: Adding query filter params (type=network) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:01.086391 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0abefcc-3aa9-4763-b852-f2f9bdeb38bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:01.087753 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0abefcc-3aa9-4763-b852-f2f9bdeb38bf None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:01.097900 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 352/721] 10.222.0.42 () {56 vars in 1112 bytes} [Thu Sep 15 07:21:00 2022] GET /identity/v3/services?type=network => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:01.100228 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0abefcc-3aa9-4763-b852-f2f9bdeb38bf None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:01.135221 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572a9a61-bc4b-4d83-a0fe-cd5dde92d5f8 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:01.177437 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782adc07-1825-40f0-b9c3-01c498fc78fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:01.178416 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782adc07-1825-40f0-b9c3-01c498fc78fe None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:01.178703 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782adc07-1825-40f0-b9c3-01c498fc78fe None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:01.178983 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782adc07-1825-40f0-b9c3-01c498fc78fe None admin] PATH_INFO: `/v3/projects/service` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:01.187196 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782adc07-1825-40f0-b9c3-01c498fc78fe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:01.189352 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782adc07-1825-40f0-b9c3-01c498fc78fe None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:01.202774 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-782adc07-1825-40f0-b9c3-01c498fc78fe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 07:21:01.202774 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:01.202774 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:01.202774 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:01.202774 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:01.202774 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:01.202774 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:01.202774 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:01.202774 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:01.202774 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:01.202774 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:01.202774 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:01.202774 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 07:21:01.202774 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 07:21:01.202774 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 07:21:01.203763 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 07:21:01.203763 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:01.203763 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:01.203763 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:01.203763 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:01.203763 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:01.203763 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:01.203763 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:01.203763 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:01.203763 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:01.203763 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:01.203763 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:01.203763 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:01.203763 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:01.203763 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:01.203763 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:01.203763 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:01.204650 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 07:21:01.204650 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 07:21:01.204650 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 07:21:01.204650 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 07:21:01.204650 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 07:21:01.204650 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 07:21:01.204650 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 07:21:01.204650 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:21:01.204650 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 370/722] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 15 07:21:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:21:01.205136 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782adc07-1825-40f0-b9c3-01c498fc78fe None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:01.223784 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0abefcc-3aa9-4763-b852-f2f9bdeb38bf None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:01.264324 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc89d02-079a-4dc9-a860-44541402acaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:01.265192 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:01.265711 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc89d02-079a-4dc9-a860-44541402acaa None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:01.265915 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc89d02-079a-4dc9-a860-44541402acaa None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:01.266191 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc89d02-079a-4dc9-a860-44541402acaa None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:01.267264 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc89d02-079a-4dc9-a860-44541402acaa None admin] RBAC: Adding query filter params (name=service) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:01.267482 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc89d02-079a-4dc9-a860-44541402acaa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:01.268866 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc89d02-079a-4dc9-a860-44541402acaa None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:01.279705 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 353/723] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Sep 15 07:21:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:01.281099 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abee577-9eed-4bbe-bb4b-a3d415120f9b None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:01.281824 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abee577-9eed-4bbe-bb4b-a3d415120f9b None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:01.282005 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abee577-9eed-4bbe-bb4b-a3d415120f9b None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:01.283280 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 354/724] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:21:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:21:01.284054 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abee577-9eed-4bbe-bb4b-a3d415120f9b None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:01.323611 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782adc07-1825-40f0-b9c3-01c498fc78fe None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:01.346921 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:01.359727 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49846c65-3417-4922-be99-a04b11aa2f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:01.360589 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49846c65-3417-4922-be99-a04b11aa2f9a None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:01.361687 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49846c65-3417-4922-be99-a04b11aa2f9a None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:01.361687 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49846c65-3417-4922-be99-a04b11aa2f9a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:01.372494 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49846c65-3417-4922-be99-a04b11aa2f9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:21:01.378545 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-49846c65-3417-4922-be99-a04b11aa2f9a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 07:21:01.378545 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:01.378545 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:01.378545 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:01.378545 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:01.378545 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:01.378545 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:01.378545 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:01.378545 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:01.378545 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:01.378545 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:01.378545 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:01.378545 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:21:01.378545 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:21:01.378545 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:21:01.378545 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:21:01.379483 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:21:01.379483 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:21:01.379483 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:21:01.379483 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:21:01.379483 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:01.379483 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:01.379483 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:01.379483 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:01.379483 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:01.379483 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:01.379483 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:01.379483 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:01.379483 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:01.379483 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:01.379483 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:01.379483 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:01.380409 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:01.380409 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:01.380409 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:01.380409 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:01.380409 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:21:01.380409 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:21:01.380409 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:21:01.380409 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:21:01.380409 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:21:01.380409 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:21:01.380409 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 07:21:01.380409 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:21:01.380409 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 371/725] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 15 07:21:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:21:01.385271 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49846c65-3417-4922-be99-a04b11aa2f9a None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:01.413267 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abee577-9eed-4bbe-bb4b-a3d415120f9b None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:01.452194 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611471f3-8d23-4ce4-80ac-08ec22b66c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:01.452866 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611471f3-8d23-4ce4-80ac-08ec22b66c40 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:01.453172 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611471f3-8d23-4ce4-80ac-08ec22b66c40 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:01.453421 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611471f3-8d23-4ce4-80ac-08ec22b66c40 None admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:01.454147 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611471f3-8d23-4ce4-80ac-08ec22b66c40 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:01.454457 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611471f3-8d23-4ce4-80ac-08ec22b66c40 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:01.455887 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611471f3-8d23-4ce4-80ac-08ec22b66c40 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:01.470260 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 355/726] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 15 07:21:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:01.470260 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c628522a-a06f-4099-812b-1f5cd255640d None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:01.470260 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c628522a-a06f-4099-812b-1f5cd255640d None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:01.470260 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c628522a-a06f-4099-812b-1f5cd255640d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:01.491949 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-c628522a-a06f-4099-812b-1f5cd255640d None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:21:01.493229 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-c628522a-a06f-4099-812b-1f5cd255640d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:21:01.495892 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49846c65-3417-4922-be99-a04b11aa2f9a None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:01.509293 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:01.530086 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5d7f8b-6684-4909-b8b9-2bd4f88aac43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:01.530730 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5d7f8b-6684-4909-b8b9-2bd4f88aac43 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:01.531625 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5d7f8b-6684-4909-b8b9-2bd4f88aac43 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:01.532038 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5d7f8b-6684-4909-b8b9-2bd4f88aac43 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:01.542764 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5d7f8b-6684-4909-b8b9-2bd4f88aac43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:01.544232 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5d7f8b-6684-4909-b8b9-2bd4f88aac43 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:01.560294 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-ed5d7f8b-6684-4909-b8b9-2bd4f88aac43 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:21:01.560294 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:01.560294 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:01.560294 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:01.560294 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:01.560294 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:01.560294 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:01.560294 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:01.560294 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:01.560294 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:01.560294 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:01.560294 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:01.560294 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 07:21:01.560294 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 07:21:01.560294 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 07:21:01.560294 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 07:21:01.561404 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:01.561404 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:01.561404 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:01.561404 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:01.561404 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:01.561404 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:01.561404 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:01.561404 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:01.561404 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:01.561404 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:01.561404 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:01.561404 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:01.561404 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:01.561404 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:01.561404 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:01.561404 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:01.561404 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 07:21:01.562394 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 07:21:01.562394 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:21:01.562394 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:21:01.562394 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 372/727] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:21:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:21:01.562394 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5d7f8b-6684-4909-b8b9-2bd4f88aac43 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:01.573541 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 356/728] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:21:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 07:21:01.574707 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c628522a-a06f-4099-812b-1f5cd255640d None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:01.616294 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:01.695485 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5d7f8b-6684-4909-b8b9-2bd4f88aac43 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:01.698807 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c628522a-a06f-4099-812b-1f5cd255640d None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:01.731886 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e05bfcc-bd68-40ff-94d0-d6c132fdd212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:01.732910 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e05bfcc-bd68-40ff-94d0-d6c132fdd212 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:01.733399 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e05bfcc-bd68-40ff-94d0-d6c132fdd212 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:01.733924 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e05bfcc-bd68-40ff-94d0-d6c132fdd212 None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:01.735578 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e05bfcc-bd68-40ff-94d0-d6c132fdd212 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:01.736098 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e05bfcc-bd68-40ff-94d0-d6c132fdd212 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:01.737773 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e05bfcc-bd68-40ff-94d0-d6c132fdd212 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:01.737960 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9ba99f-e486-4b13-8795-3c5ff4dde550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:01.738530 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9ba99f-e486-4b13-8795-3c5ff4dde550 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:01.738809 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9ba99f-e486-4b13-8795-3c5ff4dde550 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:01.739079 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9ba99f-e486-4b13-8795-3c5ff4dde550 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:01.739807 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9ba99f-e486-4b13-8795-3c5ff4dde550 None admin] RBAC: Adding query filter params (role.id=6bbb643e0d8d4bc0a9b27c26edf59216, scope.project.id=4e9987720930473da840579654311bc2, user.id=a080c7ab9b684abca6828067f2572c3b) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:01.740112 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9ba99f-e486-4b13-8795-3c5ff4dde550 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:01.741820 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9ba99f-e486-4b13-8795-3c5ff4dde550 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:01.747780 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 357/729] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 15 07:21:01 2022] GET /identity/v3/role_assignments?user.id=a080c7ab9b684abca6828067f2572c3b&scope.project.id=4e9987720930473da840579654311bc2&role.id=6bbb643e0d8d4bc0a9b27c26edf59216 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:01.748384 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9ba99f-e486-4b13-8795-3c5ff4dde550 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:01.750148 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 373/730] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 15 07:21:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:01.762789 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e05bfcc-bd68-40ff-94d0-d6c132fdd212 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:01.831552 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:01.886663 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:01.891871 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9ba99f-e486-4b13-8795-3c5ff4dde550 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:01.943036 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c442cf3-1514-420b-af98-72ca921a0ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:01.943873 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c442cf3-1514-420b-af98-72ca921a0ffc None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:01.944160 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c442cf3-1514-420b-af98-72ca921a0ffc None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:01.944434 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c442cf3-1514-420b-af98-72ca921a0ffc None admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:01.945094 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c442cf3-1514-420b-af98-72ca921a0ffc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:01.945395 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c442cf3-1514-420b-af98-72ca921a0ffc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:01.948757 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c442cf3-1514-420b-af98-72ca921a0ffc None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:01.962546 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 358/731] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 15 07:21:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:01.963503 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c442cf3-1514-420b-af98-72ca921a0ffc None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:01.992629 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e05bfcc-bd68-40ff-94d0-d6c132fdd212 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:02.032168 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b12155d-49af-4bf8-9fa6-9f880d092d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:02.032736 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b12155d-49af-4bf8-9fa6-9f880d092d50 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:02.032946 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b12155d-49af-4bf8-9fa6-9f880d092d50 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:02.033221 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b12155d-49af-4bf8-9fa6-9f880d092d50 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:02.045021 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b12155d-49af-4bf8-9fa6-9f880d092d50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:02.048041 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b12155d-49af-4bf8-9fa6-9f880d092d50 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:02.067025 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-9b12155d-49af-4bf8-9fa6-9f880d092d50 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:21:02.067025 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:02.067025 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:02.067025 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:02.067025 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:02.067025 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:02.067025 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:02.067025 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:02.067025 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:02.067025 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:02.067025 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:02.067025 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:02.067025 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 07:21:02.067025 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 07:21:02.067025 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 07:21:02.067025 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 07:21:02.067988 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:02.067988 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:02.067988 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:02.067988 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:02.067988 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:02.067988 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:02.067988 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:02.067988 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:02.067988 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:02.067988 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:02.067988 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:02.067988 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:02.067988 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:02.067988 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:02.067988 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:02.067988 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:02.067988 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 07:21:02.068844 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 07:21:02.068844 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:21:02.068844 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:21:02.072922 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 374/732] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:21:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:21:02.073985 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b12155d-49af-4bf8-9fa6-9f880d092d50 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:02.085551 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c442cf3-1514-420b-af98-72ca921a0ffc None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:02.122145 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c44340-d7f9-4cb1-b2d8-2a6be6c4b710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:02.123125 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c44340-d7f9-4cb1-b2d8-2a6be6c4b710 None admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:02.123608 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c44340-d7f9-4cb1-b2d8-2a6be6c4b710 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:02.124093 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c44340-d7f9-4cb1-b2d8-2a6be6c4b710 None admin] PATH_INFO: `/v3/services` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:02.125690 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c44340-d7f9-4cb1-b2d8-2a6be6c4b710 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:02.127389 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c44340-d7f9-4cb1-b2d8-2a6be6c4b710 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:02.170014 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:02.193536 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b12155d-49af-4bf8-9fa6-9f880d092d50 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:02.228007 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f257ff-1a28-4cfc-9744-da78d2785c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:02.228682 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f257ff-1a28-4cfc-9744-da78d2785c11 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:02.228997 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f257ff-1a28-4cfc-9744-da78d2785c11 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:02.229372 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f257ff-1a28-4cfc-9744-da78d2785c11 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:02.238712 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f257ff-1a28-4cfc-9744-da78d2785c11 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:02.240229 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f257ff-1a28-4cfc-9744-da78d2785c11 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:02.244748 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 359/733] 10.222.0.42 () {60 vars in 1141 bytes} [Thu Sep 15 07:21:01 2022] POST /identity/v3/services => generated 251 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:21:02.245967 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c44340-d7f9-4cb1-b2d8-2a6be6c4b710 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:02.253520 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-e5f257ff-1a28-4cfc-9744-da78d2785c11 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 15 07:21:02.253520 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:02.253520 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:02.253520 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:02.253520 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:02.253520 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:02.253520 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:02.253520 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:02.253520 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:02.253520 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:02.253520 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:02.253520 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:02.253520 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 07:21:02.253520 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 07:21:02.253520 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 07:21:02.253520 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 07:21:02.254553 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:02.254553 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:02.254553 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 07:21:02.254553 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 07:21:02.254553 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 07:21:02.254553 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 07:21:02.254553 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:02.254553 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:02.254553 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:02.254553 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:02.254553 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:02.254553 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:02.254553 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:02.254553 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:02.254553 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:02.254553 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:02.254553 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:02.254553 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:02.255502 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:02.255502 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:02.255502 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 07:21:02.255502 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 07:21:02.255502 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 07:21:02.255502 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 07:21:02.255502 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 07:21:02.255502 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 07:21:02.255502 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 15 07:21:02.255502 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:21:02.255502 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 375/734] 10.222.0.42 () {56 vars in 1117 bytes} [Thu Sep 15 07:21:02 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:21:02.267229 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f257ff-1a28-4cfc-9744-da78d2785c11 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:02.280179 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:02.405554 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c44340-d7f9-4cb1-b2d8-2a6be6c4b710 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:02.450023 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c76d9c7-6176-4ec4-8132-430cc198fa82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:02.451036 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c76d9c7-6176-4ec4-8132-430cc198fa82 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:02.452150 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c76d9c7-6176-4ec4-8132-430cc198fa82 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:02.452636 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c76d9c7-6176-4ec4-8132-430cc198fa82 None admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:02.453681 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c76d9c7-6176-4ec4-8132-430cc198fa82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:02.454225 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c76d9c7-6176-4ec4-8132-430cc198fa82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:02.455909 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c76d9c7-6176-4ec4-8132-430cc198fa82 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:02.472538 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 360/735] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 15 07:21:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:02.479452 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c76d9c7-6176-4ec4-8132-430cc198fa82 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:02.502005 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f257ff-1a28-4cfc-9744-da78d2785c11 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:02.553827 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88937992-6e29-4620-9017-1da112ac9490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:02.555014 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88937992-6e29-4620-9017-1da112ac9490 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:02.556333 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88937992-6e29-4620-9017-1da112ac9490 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:02.556837 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88937992-6e29-4620-9017-1da112ac9490 None admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:02.557794 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88937992-6e29-4620-9017-1da112ac9490 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:02.558334 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88937992-6e29-4620-9017-1da112ac9490 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:02.562024 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88937992-6e29-4620-9017-1da112ac9490 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:02.573225 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:02.591743 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 376/736] 10.222.0.42 () {56 vars in 1149 bytes} [Thu Sep 15 07:21:02 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:02.592781 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88937992-6e29-4620-9017-1da112ac9490 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:02.610206 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:02.646223 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c76d9c7-6176-4ec4-8132-430cc198fa82 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:02.689514 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb895f4-78a3-4a97-a062-af5f97009b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:02.690504 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb895f4-78a3-4a97-a062-af5f97009b81 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:02.690992 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb895f4-78a3-4a97-a062-af5f97009b81 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:02.691487 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb895f4-78a3-4a97-a062-af5f97009b81 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:02.700075 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb895f4-78a3-4a97-a062-af5f97009b81 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:02.701779 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb895f4-78a3-4a97-a062-af5f97009b81 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:02.716949 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-7fb895f4-78a3-4a97-a062-af5f97009b81 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 15 07:21:02.716949 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:02.716949 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:02.716949 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:02.716949 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:02.716949 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:02.716949 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:02.716949 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:02.716949 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:02.716949 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:02.716949 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:02.716949 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:02.716949 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 07:21:02.716949 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 07:21:02.716949 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 07:21:02.716949 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 07:21:02.718022 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:02.718022 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:02.718022 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 07:21:02.718022 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 07:21:02.718022 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 07:21:02.718022 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 07:21:02.718022 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:02.718022 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:02.718022 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:02.718022 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:02.718022 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:02.718022 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:02.718022 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:02.718022 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:02.718022 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:02.718022 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:02.718022 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:02.718022 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:02.719199 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:02.719199 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:02.719199 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 07:21:02.719199 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 07:21:02.719199 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 07:21:02.719199 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 07:21:02.719199 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 07:21:02.719199 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 07:21:02.719199 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 15 07:21:02.719199 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:21:02.719741 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 361/737] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 15 07:21:02 2022] GET /identity/v3/users/swift => generated 83 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 07:21:02.725296 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb895f4-78a3-4a97-a062-af5f97009b81 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:02.726583 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88937992-6e29-4620-9017-1da112ac9490 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:02.768518 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b33094-570b-4f1a-a97b-56e12cb98098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:02.769281 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b33094-570b-4f1a-a97b-56e12cb98098 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:02.769609 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b33094-570b-4f1a-a97b-56e12cb98098 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:02.769944 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b33094-570b-4f1a-a97b-56e12cb98098 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:02.840322 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb895f4-78a3-4a97-a062-af5f97009b81 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:02.840737 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:02.849008 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b33094-570b-4f1a-a97b-56e12cb98098 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:02.850346 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b33094-570b-4f1a-a97b-56e12cb98098 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:02.879493 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:02.885457 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a58b407-3759-422d-9cff-43f5c38f1641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:02.886365 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a58b407-3759-422d-9cff-43f5c38f1641 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:02.886850 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a58b407-3759-422d-9cff-43f5c38f1641 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:02.887386 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a58b407-3759-422d-9cff-43f5c38f1641 None admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:02.888349 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a58b407-3759-422d-9cff-43f5c38f1641 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:02.889011 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a58b407-3759-422d-9cff-43f5c38f1641 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:02.892067 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a58b407-3759-422d-9cff-43f5c38f1641 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:02.915005 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 362/738] 10.222.0.42 () {56 vars in 1135 bytes} [Thu Sep 15 07:21:02 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:02.923398 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a58b407-3759-422d-9cff-43f5c38f1641 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:02.979252 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 377/739] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 15 07:21:02 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 15 07:21:02.985680 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b33094-570b-4f1a-a97b-56e12cb98098 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:03.091955 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:03.100309 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b33094-570b-4f1a-a97b-56e12cb98098 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:03.150522 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f75e288-b155-467e-9cfc-528c2e04b19a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:03.151172 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f75e288-b155-467e-9cfc-528c2e04b19a None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:03.151563 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f75e288-b155-467e-9cfc-528c2e04b19a None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:03.151846 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f75e288-b155-467e-9cfc-528c2e04b19a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:03.155119 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a58b407-3759-422d-9cff-43f5c38f1641 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:03.176631 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f75e288-b155-467e-9cfc-528c2e04b19a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:03.176631 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f75e288-b155-467e-9cfc-528c2e04b19a None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:03.193097 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60d79b7-0a57-4d7c-9772-a98dac68bbea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:03.193720 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60d79b7-0a57-4d7c-9772-a98dac68bbea None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:03.198010 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60d79b7-0a57-4d7c-9772-a98dac68bbea None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:03.198010 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60d79b7-0a57-4d7c-9772-a98dac68bbea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:03.199819 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-4f75e288-b155-467e-9cfc-528c2e04b19a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:21:03.199819 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:03.199819 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:03.199819 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:03.199819 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:03.199819 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:03.199819 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:03.199819 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:03.199819 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:03.199819 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:03.199819 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:03.199819 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:03.199819 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 07:21:03.199819 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 07:21:03.199819 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 07:21:03.199819 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 07:21:03.200909 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:03.200909 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:03.200909 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:03.200909 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:03.200909 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:03.200909 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:03.200909 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:03.200909 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:03.200909 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:03.200909 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:03.200909 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:03.200909 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:03.200909 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:03.200909 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:03.200909 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:03.200909 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:03.200909 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 07:21:03.201910 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 07:21:03.201910 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:21:03.201910 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:21:03.210497 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 378/740] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:21:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:21:03.211420 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f75e288-b155-467e-9cfc-528c2e04b19a None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:03.305389 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:03.317638 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60d79b7-0a57-4d7c-9772-a98dac68bbea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:03.317638 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60d79b7-0a57-4d7c-9772-a98dac68bbea None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:03.404524 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f75e288-b155-467e-9cfc-528c2e04b19a None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:03.423863 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 363/741] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 15 07:21:02 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 07:21:03.430611 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60d79b7-0a57-4d7c-9772-a98dac68bbea None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:03.482722 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e6e55f-e22f-42a2-a332-e1a84a1e4971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:03.483571 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e6e55f-e22f-42a2-a332-e1a84a1e4971 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:03.483999 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e6e55f-e22f-42a2-a332-e1a84a1e4971 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:03.485030 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e6e55f-e22f-42a2-a332-e1a84a1e4971 None admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:03.485911 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e6e55f-e22f-42a2-a332-e1a84a1e4971 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:03.486388 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e6e55f-e22f-42a2-a332-e1a84a1e4971 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:03.488079 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e6e55f-e22f-42a2-a332-e1a84a1e4971 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:03.503791 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 379/742] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 15 07:21:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:03.510214 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0613cb9e-4beb-402a-a1f2-857c3afc0858 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:03.510856 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0613cb9e-4beb-402a-a1f2-857c3afc0858 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:03.511435 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0613cb9e-4beb-402a-a1f2-857c3afc0858 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:03.513166 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 380/743] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:21:03 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:21:03.514095 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0613cb9e-4beb-402a-a1f2-857c3afc0858 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:03.528406 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:03.547921 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:03.559243 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60d79b7-0a57-4d7c-9772-a98dac68bbea None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:03.602212 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c839388-a489-4320-b52d-b900837abb8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:03.603208 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c839388-a489-4320-b52d-b900837abb8b None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:03.603692 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c839388-a489-4320-b52d-b900837abb8b None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:03.604242 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c839388-a489-4320-b52d-b900837abb8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:03.617194 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c839388-a489-4320-b52d-b900837abb8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:03.619686 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c839388-a489-4320-b52d-b900837abb8b None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:03.636411 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0613cb9e-4beb-402a-a1f2-857c3afc0858 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:03.637920 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-6c839388-a489-4320-b52d-b900837abb8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:21:03.637920 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:03.637920 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:03.637920 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:03.637920 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:03.637920 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:03.637920 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:03.637920 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:03.637920 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:03.637920 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:03.637920 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:03.637920 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:03.637920 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 07:21:03.637920 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 07:21:03.637920 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 07:21:03.637920 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 07:21:03.638839 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:03.638839 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:03.638839 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:03.638839 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:03.638839 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:03.638839 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:03.638839 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:03.638839 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:03.638839 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:03.638839 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:03.638839 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:03.638839 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:03.638839 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:03.638839 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:03.638839 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:03.638839 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:03.638839 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 07:21:03.639730 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 07:21:03.639730 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:21:03.639730 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:21:03.642637 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 364/744] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:21:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:21:03.644075 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552105f-573d-423e-a15f-6a56c0ffce09 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:03.644578 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552105f-573d-423e-a15f-6a56c0ffce09 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:03.645103 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552105f-573d-423e-a15f-6a56c0ffce09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:03.666505 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-8552105f-573d-423e-a15f-6a56c0ffce09 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:21:03.668053 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-8552105f-573d-423e-a15f-6a56c0ffce09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:21:03.674995 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc39302f-3b1a-4335-af11-0686e58d5a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:03.675678 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc39302f-3b1a-4335-af11-0686e58d5a4f None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:03.675956 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc39302f-3b1a-4335-af11-0686e58d5a4f None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:03.676226 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc39302f-3b1a-4335-af11-0686e58d5a4f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:03.683761 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc39302f-3b1a-4335-af11-0686e58d5a4f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:03.685721 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc39302f-3b1a-4335-af11-0686e58d5a4f None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:03.698856 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-cc39302f-3b1a-4335-af11-0686e58d5a4f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 07:21:03.698856 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:03.698856 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:03.698856 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:03.698856 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:03.698856 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:03.698856 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:03.698856 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:03.698856 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:03.698856 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:03.698856 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:03.698856 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:03.698856 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 07:21:03.698856 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 07:21:03.698856 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 07:21:03.699777 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 07:21:03.699777 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:03.699777 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:03.699777 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:03.699777 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:03.699777 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:03.699777 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:03.699777 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:03.699777 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:03.699777 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:03.699777 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:03.699777 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:03.699777 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:03.699777 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:03.699777 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:03.699777 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:03.699777 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:03.700639 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 07:21:03.700639 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 07:21:03.700639 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 07:21:03.700639 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 07:21:03.700639 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 07:21:03.700639 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 07:21:03.700639 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 07:21:03.700639 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:21:03.704924 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 381/745] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 15 07:21:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:21:03.705920 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc39302f-3b1a-4335-af11-0686e58d5a4f None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:03.753056 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 365/746] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:21:03 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 07:21:03.754315 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8552105f-573d-423e-a15f-6a56c0ffce09 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:03.806801 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:03.841944 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:03.861030 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc39302f-3b1a-4335-af11-0686e58d5a4f None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:03.882051 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8552105f-573d-423e-a15f-6a56c0ffce09 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:03.921275 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddb3eb8-0ce3-4230-8ea3-dfe95a68e236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:03.924000 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb3eb8-0ce3-4230-8ea3-dfe95a68e236 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:03.924396 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb3eb8-0ce3-4230-8ea3-dfe95a68e236 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:03.924767 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb3eb8-0ce3-4230-8ea3-dfe95a68e236 None admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:03.925571 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddb3eb8-0ce3-4230-8ea3-dfe95a68e236 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:03.925999 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddb3eb8-0ce3-4230-8ea3-dfe95a68e236 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:03.926191 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad857f44-6dc6-4056-9ad2-75468f3cb1aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:03.926953 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad857f44-6dc6-4056-9ad2-75468f3cb1aa None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:03.927373 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad857f44-6dc6-4056-9ad2-75468f3cb1aa None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:03.932655 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddb3eb8-0ce3-4230-8ea3-dfe95a68e236 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:03.933158 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad857f44-6dc6-4056-9ad2-75468f3cb1aa None admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:03.933978 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad857f44-6dc6-4056-9ad2-75468f3cb1aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:03.934429 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad857f44-6dc6-4056-9ad2-75468f3cb1aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:03.936311 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad857f44-6dc6-4056-9ad2-75468f3cb1aa None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:03.941949 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 366/747] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 15 07:21:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:03.955231 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddb3eb8-0ce3-4230-8ea3-dfe95a68e236 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:03.964720 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 382/748] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 15 07:21:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:03.970769 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad857f44-6dc6-4056-9ad2-75468f3cb1aa None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:04.136741 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad857f44-6dc6-4056-9ad2-75468f3cb1aa None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:04.138676 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddb3eb8-0ce3-4230-8ea3-dfe95a68e236 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:04.171182 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:04.174712 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:04.179327 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1724be7-bbf4-4827-a621-7d3887d389fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:04.179717 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6401f2-4493-4f38-9042-f8610415c26d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:04.179976 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1724be7-bbf4-4827-a621-7d3887d389fd None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:04.180256 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1724be7-bbf4-4827-a621-7d3887d389fd None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:04.180352 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6401f2-4493-4f38-9042-f8610415c26d None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:04.180530 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1724be7-bbf4-4827-a621-7d3887d389fd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:04.180627 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6401f2-4493-4f38-9042-f8610415c26d None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:04.180852 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6401f2-4493-4f38-9042-f8610415c26d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:04.181598 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6401f2-4493-4f38-9042-f8610415c26d None admin] RBAC: Adding query filter params (role.id=0e661b5a67664692b933e5da9fc2e488, scope.project.id=4e9987720930473da840579654311bc2, user.id=f9c68c4bd9074d8d9b893755e28c0807) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:04.181923 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6401f2-4493-4f38-9042-f8610415c26d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:04.183370 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6401f2-4493-4f38-9042-f8610415c26d None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:04.188237 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1724be7-bbf4-4827-a621-7d3887d389fd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:04.193946 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 367/749] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 15 07:21:03 2022] GET /identity/v3/role_assignments?user.id=f9c68c4bd9074d8d9b893755e28c0807&scope.project.id=4e9987720930473da840579654311bc2&role.id=0e661b5a67664692b933e5da9fc2e488 => generated 259 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:04.194146 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1724be7-bbf4-4827-a621-7d3887d389fd None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:04.194235 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77176a8-38dd-4417-8853-58cf92760425 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:04.194235 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77176a8-38dd-4417-8853-58cf92760425 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:04.194235 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77176a8-38dd-4417-8853-58cf92760425 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:04.194235 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 368/750] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:21:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:21:04.194235 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77176a8-38dd-4417-8853-58cf92760425 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:04.208349 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-e1724be7-bbf4-4827-a621-7d3887d389fd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 07:21:04.208349 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:04.208349 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:04.208349 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:04.208349 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:04.208349 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:04.208349 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:04.208349 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:04.208349 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:04.208349 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:04.208349 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:04.208349 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:04.208349 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 07:21:04.208349 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 07:21:04.208349 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 07:21:04.209327 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 07:21:04.209327 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:04.209327 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:04.209327 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:04.209327 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:04.209327 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:04.209327 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:04.209327 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:04.209327 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:04.209327 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:04.209327 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:04.209327 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:04.209327 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:04.209327 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:04.209327 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:04.209327 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:04.209327 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:04.210492 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 07:21:04.210492 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 07:21:04.210492 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 07:21:04.210492 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 07:21:04.210492 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 07:21:04.210492 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 07:21:04.210492 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 07:21:04.210492 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:21:04.210492 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 383/751] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 15 07:21:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 07:21:04.211215 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a906d-9718-4f74-a89f-9ef6652e91c5 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:04.211503 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a906d-9718-4f74-a89f-9ef6652e91c5 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:04.211779 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a906d-9718-4f74-a89f-9ef6652e91c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:04.232990 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-9e1a906d-9718-4f74-a89f-9ef6652e91c5 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:21:04.234799 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-9e1a906d-9718-4f74-a89f-9ef6652e91c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:21:04.318984 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77176a8-38dd-4417-8853-58cf92760425 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:04.329749 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 384/752] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:21:04 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 07:21:04.333078 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1a906d-9718-4f74-a89f-9ef6652e91c5 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:04.385481 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113de147-422a-4984-9a8f-21eea4d0fabe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:04.386538 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113de147-422a-4984-9a8f-21eea4d0fabe None admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:04.387087 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113de147-422a-4984-9a8f-21eea4d0fabe None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:04.387571 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113de147-422a-4984-9a8f-21eea4d0fabe None admin] PATH_INFO: `/v3/services` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:04.388522 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113de147-422a-4984-9a8f-21eea4d0fabe None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:04.390157 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113de147-422a-4984-9a8f-21eea4d0fabe None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:04.472299 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 369/753] 10.222.0.42 () {60 vars in 1141 bytes} [Thu Sep 15 07:21:04 2022] POST /identity/v3/services => generated 240 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:21:04.478411 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1a906d-9718-4f74-a89f-9ef6652e91c5 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:04.500609 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:04.558347 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f55935-97d9-4034-b6f3-f2f49b0a9349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:04.559382 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f55935-97d9-4034-b6f3-f2f49b0a9349 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:04.559898 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f55935-97d9-4034-b6f3-f2f49b0a9349 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:04.560641 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f55935-97d9-4034-b6f3-f2f49b0a9349 None admin] PATH_INFO: `/v3/projects` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:04.561537 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f55935-97d9-4034-b6f3-f2f49b0a9349 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:04.562104 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f55935-97d9-4034-b6f3-f2f49b0a9349 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:04.564371 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f55935-97d9-4034-b6f3-f2f49b0a9349 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:04.589395 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 385/754] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 15 07:21:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:04.590818 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113de147-422a-4984-9a8f-21eea4d0fabe None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:04.653430 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f55935-97d9-4034-b6f3-f2f49b0a9349 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:04.814090 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f55935-97d9-4034-b6f3-f2f49b0a9349 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:04.820675 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:04.866541 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d40349-a070-4e9d-8ea8-5d120948305b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:04.867254 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d40349-a070-4e9d-8ea8-5d120948305b None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:04.867521 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d40349-a070-4e9d-8ea8-5d120948305b None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:04.867799 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d40349-a070-4e9d-8ea8-5d120948305b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:04.883593 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113de147-422a-4984-9a8f-21eea4d0fabe None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:04.894906 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d40349-a070-4e9d-8ea8-5d120948305b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:21:04.903216 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-48d40349-a070-4e9d-8ea8-5d120948305b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 07:21:04.903216 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:04.903216 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:04.903216 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:04.903216 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:04.903216 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:04.903216 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:04.903216 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:04.903216 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:04.903216 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:04.903216 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:04.903216 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:04.903216 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:21:04.903216 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:21:04.903216 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:21:04.903216 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:21:04.904185 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:21:04.904185 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:21:04.904185 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:21:04.904185 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:21:04.904185 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:04.904185 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:04.904185 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:04.904185 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:04.904185 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:04.904185 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:04.904185 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:04.904185 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:04.904185 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:04.904185 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:04.904185 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:04.904185 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:04.905026 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:04.905026 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:04.905026 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:04.905026 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:04.905026 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:21:04.905026 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:21:04.905026 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:21:04.905026 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:21:04.905026 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:21:04.905026 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:21:04.905026 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 07:21:04.905026 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:21:04.905026 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 386/755] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 15 07:21:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 07:21:04.912235 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d40349-a070-4e9d-8ea8-5d120948305b None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:04.961152 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b376225-1918-4a24-9609-0767df09f767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:04.961796 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b376225-1918-4a24-9609-0767df09f767 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:04.962073 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b376225-1918-4a24-9609-0767df09f767 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:04.962346 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b376225-1918-4a24-9609-0767df09f767 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:04.963075 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b376225-1918-4a24-9609-0767df09f767 None admin] RBAC: Adding query filter params (role.id=6bbb643e0d8d4bc0a9b27c26edf59216, scope.project.id=4e9987720930473da840579654311bc2, user.id=72b90d35087e4d7a80e9417082a1541f) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:04.963377 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b376225-1918-4a24-9609-0767df09f767 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:04.964801 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b376225-1918-4a24-9609-0767df09f767 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:04.977702 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 370/756] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 15 07:21:04 2022] GET /identity/v3/role_assignments?user.id=72b90d35087e4d7a80e9417082a1541f&scope.project.id=4e9987720930473da840579654311bc2&role.id=6bbb643e0d8d4bc0a9b27c26edf59216 => generated 259 bytes in 387 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:04.982157 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:05.057432 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d40349-a070-4e9d-8ea8-5d120948305b None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:05.103908 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d136e83-889c-4186-b3a1-b9f820146a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:05.104543 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d136e83-889c-4186-b3a1-b9f820146a14 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:05.104818 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d136e83-889c-4186-b3a1-b9f820146a14 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:05.105087 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d136e83-889c-4186-b3a1-b9f820146a14 None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:05.105792 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d136e83-889c-4186-b3a1-b9f820146a14 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:05.106098 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d136e83-889c-4186-b3a1-b9f820146a14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:05.107506 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d136e83-889c-4186-b3a1-b9f820146a14 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:05.118723 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 387/757] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 15 07:21:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:21:05.124524 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b376225-1918-4a24-9609-0767df09f767 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:05.263551 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b376225-1918-4a24-9609-0767df09f767 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:05.287585 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cd4a62-a038-47c6-a665-98b87c3af799 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:05.287911 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cd4a62-a038-47c6-a665-98b87c3af799 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:05.288124 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cd4a62-a038-47c6-a665-98b87c3af799 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:05.289440 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 388/758] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:21:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 07:21:05.290950 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:05.295250 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcde206-f8ed-403e-a160-dc97d84e43f7 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:05.295545 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcde206-f8ed-403e-a160-dc97d84e43f7 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:05.295829 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcde206-f8ed-403e-a160-dc97d84e43f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:05.308270 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328a30ec-b19f-4e55-ac90-e6bf3627ebf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:05.309266 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328a30ec-b19f-4e55-ac90-e6bf3627ebf0 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:05.309813 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328a30ec-b19f-4e55-ac90-e6bf3627ebf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:05.310588 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328a30ec-b19f-4e55-ac90-e6bf3627ebf0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:05.325675 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-5dcde206-f8ed-403e-a160-dc97d84e43f7 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:21:05.329598 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-5dcde206-f8ed-403e-a160-dc97d84e43f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:21:05.329721 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328a30ec-b19f-4e55-ac90-e6bf3627ebf0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:05.329957 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328a30ec-b19f-4e55-ac90-e6bf3627ebf0 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:05.355912 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-328a30ec-b19f-4e55-ac90-e6bf3627ebf0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:21:05.355912 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:05.355912 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:05.355912 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:05.355912 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:05.355912 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:05.355912 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:05.355912 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:05.355912 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:05.355912 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:05.355912 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:05.355912 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:05.355912 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 07:21:05.355912 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 07:21:05.355912 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 07:21:05.355912 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 07:21:05.356894 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:05.356894 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:05.356894 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:05.356894 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:05.356894 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:05.356894 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:05.356894 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:05.356894 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:05.356894 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:05.356894 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:05.356894 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:05.356894 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:05.356894 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:05.356894 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:05.356894 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:05.356894 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:05.356894 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 07:21:05.357819 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 07:21:05.357819 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:21:05.357819 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:21:05.357819 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 371/759] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:21:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:21:05.368431 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328a30ec-b19f-4e55-ac90-e6bf3627ebf0 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:05.479189 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 389/760] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:21:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 07:21:05.510354 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328a30ec-b19f-4e55-ac90-e6bf3627ebf0 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:05.552626 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86e6a27-d32e-412a-9075-610a9c156d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:05.553243 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86e6a27-d32e-412a-9075-610a9c156d3e None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:05.553519 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86e6a27-d32e-412a-9075-610a9c156d3e None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:05.553818 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86e6a27-d32e-412a-9075-610a9c156d3e None admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:05.554466 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86e6a27-d32e-412a-9075-610a9c156d3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:05.554764 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86e6a27-d32e-412a-9075-610a9c156d3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:05.556124 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86e6a27-d32e-412a-9075-610a9c156d3e None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:05.568891 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 372/761] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 15 07:21:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:05.574125 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcde206-f8ed-403e-a160-dc97d84e43f7 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:05.624993 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:05.713232 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcde206-f8ed-403e-a160-dc97d84e43f7 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:05.749083 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975527fa-9f4f-40e8-b4c2-64a745e7cec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:05.749732 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975527fa-9f4f-40e8-b4c2-64a745e7cec3 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:05.750088 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975527fa-9f4f-40e8-b4c2-64a745e7cec3 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:05.750364 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975527fa-9f4f-40e8-b4c2-64a745e7cec3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:05.757518 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975527fa-9f4f-40e8-b4c2-64a745e7cec3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:05.758987 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975527fa-9f4f-40e8-b4c2-64a745e7cec3 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:05.771419 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-975527fa-9f4f-40e8-b4c2-64a745e7cec3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 15 07:21:05.771419 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:05.771419 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:05.771419 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:05.771419 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:05.771419 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:05.771419 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:05.771419 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:05.771419 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:05.771419 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:05.771419 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:05.771419 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:05.771419 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 07:21:05.771419 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 07:21:05.771419 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 07:21:05.771419 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 07:21:05.772391 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:05.772391 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:05.772391 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 07:21:05.772391 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 07:21:05.772391 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 07:21:05.772391 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 07:21:05.772391 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:05.772391 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:05.772391 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:05.772391 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:05.772391 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:05.772391 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:05.772391 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:05.772391 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:05.772391 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:05.772391 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:05.772391 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:05.772391 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:05.773346 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:05.773346 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:05.773346 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 07:21:05.773346 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 07:21:05.773346 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 07:21:05.773346 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 07:21:05.773346 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 07:21:05.773346 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 07:21:05.773346 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 15 07:21:05.773346 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:21:05.773346 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 390/762] 10.222.0.42 () {56 vars in 1093 bytes} [Thu Sep 15 07:21:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:21:05.778129 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86e6a27-d32e-412a-9075-610a9c156d3e None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:05.805652 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975527fa-9f4f-40e8-b4c2-64a745e7cec3 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:05.841955 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:05.913147 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86e6a27-d32e-412a-9075-610a9c156d3e None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:05.979714 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902d9cf8-226b-4ad8-a9d2-5e96bde4ce20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:05.980329 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902d9cf8-226b-4ad8-a9d2-5e96bde4ce20 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:05.980610 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902d9cf8-226b-4ad8-a9d2-5e96bde4ce20 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:05.980879 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902d9cf8-226b-4ad8-a9d2-5e96bde4ce20 None admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:05.981620 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902d9cf8-226b-4ad8-a9d2-5e96bde4ce20 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:05.981926 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902d9cf8-226b-4ad8-a9d2-5e96bde4ce20 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:05.983329 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902d9cf8-226b-4ad8-a9d2-5e96bde4ce20 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:05.998616 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:06.019177 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 373/763] 10.222.0.42 () {56 vars in 1133 bytes} [Thu Sep 15 07:21:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:06.022509 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902d9cf8-226b-4ad8-a9d2-5e96bde4ce20 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:06.030956 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975527fa-9f4f-40e8-b4c2-64a745e7cec3 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:06.073013 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b601fa7-d3b6-49b4-9089-72fac35ea29a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:06.079226 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b601fa7-d3b6-49b4-9089-72fac35ea29a None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:06.081224 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b601fa7-d3b6-49b4-9089-72fac35ea29a None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:06.081507 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b601fa7-d3b6-49b4-9089-72fac35ea29a None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:06.082135 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b601fa7-d3b6-49b4-9089-72fac35ea29a None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:06.083521 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b601fa7-d3b6-49b4-9089-72fac35ea29a None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:06.098045 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-2b601fa7-d3b6-49b4-9089-72fac35ea29a None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 15 07:21:06.098045 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:06.098045 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 07:21:06.098045 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 07:21:06.098045 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 07:21:06.098045 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 07:21:06.098045 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 07:21:06.098045 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 07:21:06.098045 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 15 07:21:06.098045 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:21:06.098045 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 07:21:06.098045 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:21:06.098045 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:06.098045 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:06.098045 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:06.098045 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:06.098045 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:06.099098 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:06.099098 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:06.099098 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:06.099098 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:06.099098 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:06.099098 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:06.099098 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 07:21:06.099098 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 07:21:06.099098 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 07:21:06.099098 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 07:21:06.099098 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:06.099098 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:06.099098 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:06.099098 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:06.099098 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:06.099098 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:06.099936 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:06.099936 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:06.099936 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:06.099936 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:06.099936 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:06.099936 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:06.099936 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:06.099936 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:06.099936 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:06.099936 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:06.099936 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 07:21:06.099936 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 07:21:06.099936 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 15 07:21:06.099936 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:21:06.099936 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 391/764] 10.222.0.42 () {56 vars in 1129 bytes} [Thu Sep 15 07:21:05 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 07:21:06.110808 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b601fa7-d3b6-49b4-9089-72fac35ea29a None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:06.152959 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:06.164268 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902d9cf8-226b-4ad8-a9d2-5e96bde4ce20 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:06.202494 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59879cf3-1dec-43ac-9558-55bdbb624bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:06.203301 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59879cf3-1dec-43ac-9558-55bdbb624bae None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:06.203575 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59879cf3-1dec-43ac-9558-55bdbb624bae None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:06.203848 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59879cf3-1dec-43ac-9558-55bdbb624bae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:06.214906 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59879cf3-1dec-43ac-9558-55bdbb624bae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:06.216353 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59879cf3-1dec-43ac-9558-55bdbb624bae None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:06.232950 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-59879cf3-1dec-43ac-9558-55bdbb624bae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:21:06.232950 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:06.232950 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:06.232950 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:06.232950 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:06.232950 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:06.232950 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:06.232950 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:06.232950 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:06.232950 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:06.232950 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:06.232950 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:06.232950 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 07:21:06.232950 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 07:21:06.232950 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 07:21:06.232950 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 07:21:06.233946 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:06.233946 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:06.233946 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:06.233946 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:06.233946 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:06.233946 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:06.233946 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:06.233946 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:06.233946 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:06.233946 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:06.233946 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:06.233946 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:06.233946 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:06.233946 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:06.233946 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:06.233946 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:06.233946 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 07:21:06.234988 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 07:21:06.234988 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:21:06.234988 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:21:06.234988 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 374/765] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:21:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:21:06.240292 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59879cf3-1dec-43ac-9558-55bdbb624bae None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:06.348454 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b601fa7-d3b6-49b4-9089-72fac35ea29a None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:06.349220 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59879cf3-1dec-43ac-9558-55bdbb624bae None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:06.411109 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196f4379-fda2-4544-9a43-add976de9727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:06.411719 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196f4379-fda2-4544-9a43-add976de9727 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:06.411998 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196f4379-fda2-4544-9a43-add976de9727 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:06.412269 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196f4379-fda2-4544-9a43-add976de9727 None admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:06.412915 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196f4379-fda2-4544-9a43-add976de9727 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:06.413216 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196f4379-fda2-4544-9a43-add976de9727 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:06.414597 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196f4379-fda2-4544-9a43-add976de9727 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:06.420390 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d541763-1f7b-4cd9-b2d9-18bf22edbd7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:06.420997 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d541763-1f7b-4cd9-b2d9-18bf22edbd7f None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:06.421272 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d541763-1f7b-4cd9-b2d9-18bf22edbd7f None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:06.421542 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d541763-1f7b-4cd9-b2d9-18bf22edbd7f None admin] PATH_INFO: `/v3/services` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:06.429342 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d541763-1f7b-4cd9-b2d9-18bf22edbd7f None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:06.429342 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d541763-1f7b-4cd9-b2d9-18bf22edbd7f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:06.429342 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d541763-1f7b-4cd9-b2d9-18bf22edbd7f None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:06.435934 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 375/766] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 15 07:21:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:06.441439 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196f4379-fda2-4544-9a43-add976de9727 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:06.444050 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:06.455081 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 392/767] 10.222.0.42 () {56 vars in 1124 bytes} [Thu Sep 15 07:21:06 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:06.455983 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d541763-1f7b-4cd9-b2d9-18bf22edbd7f None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:06.490622 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:06.608354 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196f4379-fda2-4544-9a43-add976de9727 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:06.641953 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d541763-1f7b-4cd9-b2d9-18bf22edbd7f None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:06.675032 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589c419e-29b9-42dd-b0f5-9cfe9a208eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:06.675689 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589c419e-29b9-42dd-b0f5-9cfe9a208eaa None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:06.675975 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589c419e-29b9-42dd-b0f5-9cfe9a208eaa None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:06.676248 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589c419e-29b9-42dd-b0f5-9cfe9a208eaa None admin] PATH_INFO: `/v3/projects/service` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:06.689966 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589c419e-29b9-42dd-b0f5-9cfe9a208eaa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:06.691413 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589c419e-29b9-42dd-b0f5-9cfe9a208eaa None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:06.714338 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-589c419e-29b9-42dd-b0f5-9cfe9a208eaa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 07:21:06.714338 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:06.714338 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:06.714338 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:06.714338 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:06.714338 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:06.714338 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:06.714338 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:06.714338 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:06.714338 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:06.714338 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:06.714338 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:06.714338 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 07:21:06.714338 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 07:21:06.714338 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 07:21:06.715266 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 07:21:06.715266 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:06.715266 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:06.715266 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:06.715266 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:06.715266 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:06.715266 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:06.715266 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:06.715266 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:06.715266 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:06.715266 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:06.715266 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:06.715266 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:06.715266 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:06.715266 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:06.715266 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:06.715266 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:06.716149 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 07:21:06.716149 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 07:21:06.716149 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 07:21:06.716149 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 07:21:06.716149 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 07:21:06.716149 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 07:21:06.716149 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 07:21:06.716149 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:21:06.717102 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 376/768] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 15 07:21:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:21:06.723322 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9503468-9dab-4c6b-b1bb-7f1f5bb801b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:06.724006 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9503468-9dab-4c6b-b1bb-7f1f5bb801b1 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:06.724333 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9503468-9dab-4c6b-b1bb-7f1f5bb801b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:06.724607 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9503468-9dab-4c6b-b1bb-7f1f5bb801b1 None admin] PATH_INFO: `/v3/services` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:06.725246 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9503468-9dab-4c6b-b1bb-7f1f5bb801b1 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:06.725610 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9503468-9dab-4c6b-b1bb-7f1f5bb801b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:06.726996 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9503468-9dab-4c6b-b1bb-7f1f5bb801b1 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:06.728625 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589c419e-29b9-42dd-b0f5-9cfe9a208eaa None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:06.741176 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 393/769] 10.222.0.42 () {56 vars in 1124 bytes} [Thu Sep 15 07:21:06 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:06.747281 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9503468-9dab-4c6b-b1bb-7f1f5bb801b1 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:06.829804 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:06.846008 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:06.868693 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9503468-9dab-4c6b-b1bb-7f1f5bb801b1 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:06.873686 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589c419e-29b9-42dd-b0f5-9cfe9a208eaa None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:06.905961 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f60799-9121-43c9-8847-f43131b58491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:06.906628 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f60799-9121-43c9-8847-f43131b58491 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:06.906908 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f60799-9121-43c9-8847-f43131b58491 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:06.907185 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f60799-9121-43c9-8847-f43131b58491 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:06.907892 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f60799-9121-43c9-8847-f43131b58491 None admin] RBAC: Adding query filter params (interface=public, service_id=ad6ee4be2e5f413ba1a0ccd186cd9693, region_id=RegionOne) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:06.908206 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f60799-9121-43c9-8847-f43131b58491 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:06.909595 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f60799-9121-43c9-8847-f43131b58491 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:06.915158 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b3f234-0308-44f6-a2de-7d5b5a755fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:06.915770 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b3f234-0308-44f6-a2de-7d5b5a755fc8 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:06.916051 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b3f234-0308-44f6-a2de-7d5b5a755fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:06.916325 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b3f234-0308-44f6-a2de-7d5b5a755fc8 None admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:06.917030 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b3f234-0308-44f6-a2de-7d5b5a755fc8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:06.917374 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b3f234-0308-44f6-a2de-7d5b5a755fc8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:06.917831 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 394/770] 10.222.0.42 () {56 vars in 1251 bytes} [Thu Sep 15 07:21:06 2022] GET /identity/v3/endpoints?service_id=ad6ee4be2e5f413ba1a0ccd186cd9693&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:21:06.919106 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61cfd49-a0d6-465a-8d63-b8423320f5dc None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:06.919396 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61cfd49-a0d6-465a-8d63-b8423320f5dc None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:06.919677 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61cfd49-a0d6-465a-8d63-b8423320f5dc None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:06.921111 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 395/771] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:21:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:21:06.923336 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b3f234-0308-44f6-a2de-7d5b5a755fc8 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:06.931336 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bb7505-c117-4636-96fb-d7ffeaf6644e None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:06.931917 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bb7505-c117-4636-96fb-d7ffeaf6644e None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:06.932398 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bb7505-c117-4636-96fb-d7ffeaf6644e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:06.942117 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 377/772] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 15 07:21:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:06.944002 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b3f234-0308-44f6-a2de-7d5b5a755fc8 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:06.962348 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-f4bb7505-c117-4636-96fb-d7ffeaf6644e None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:21:06.964170 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-f4bb7505-c117-4636-96fb-d7ffeaf6644e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:21:07.048189 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 396/773] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:21:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 07:21:07.049567 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bb7505-c117-4636-96fb-d7ffeaf6644e None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:07.083059 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b3f234-0308-44f6-a2de-7d5b5a755fc8 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:07.131156 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:07.154851 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea549d72-e37c-4915-ab76-b676b3162a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:07.155859 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea549d72-e37c-4915-ab76-b676b3162a25 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:07.157074 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea549d72-e37c-4915-ab76-b676b3162a25 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:07.157810 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea549d72-e37c-4915-ab76-b676b3162a25 None admin] PATH_INFO: `/v3/services` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:07.158638 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea549d72-e37c-4915-ab76-b676b3162a25 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:07.161888 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea549d72-e37c-4915-ab76-b676b3162a25 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:07.166339 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:07.168591 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 378/774] 10.222.0.42 () {56 vars in 1087 bytes} [Thu Sep 15 07:21:06 2022] GET /identity/v3/services => generated 772 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:07.248285 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bb7505-c117-4636-96fb-d7ffeaf6644e None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:07.303735 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1a7986-f714-4159-a1e6-2482ba5b1438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:07.304417 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a7986-f714-4159-a1e6-2482ba5b1438 None admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:07.304698 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a7986-f714-4159-a1e6-2482ba5b1438 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:07.304974 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a7986-f714-4159-a1e6-2482ba5b1438 None admin] PATH_INFO: `/v3/projects/4e9987720930473da840579654311bc2/users/a080c7ab9b684abca6828067f2572c3b/roles/6bbb643e0d8d4bc0a9b27c26edf59216` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:07.307023 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1a7986-f714-4159-a1e6-2482ba5b1438 None admin] RBAC: Authorizing `identity:create_grant(project_id=4e9987720930473da840579654311bc2, user_id=a080c7ab9b684abca6828067f2572c3b, role_id=6bbb643e0d8d4bc0a9b27c26edf59216)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:07.308472 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1a7986-f714-4159-a1e6-2482ba5b1438 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:07.382755 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60e2583-5507-4a8f-8a63-e1e9b89c4115 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:07.383401 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60e2583-5507-4a8f-8a63-e1e9b89c4115 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:07.383878 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60e2583-5507-4a8f-8a63-e1e9b89c4115 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:07.386004 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 379/775] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:21:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:21:07.393372 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f66ebb8-2ae5-4d91-b16f-056391c2afaf None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:07.394047 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f66ebb8-2ae5-4d91-b16f-056391c2afaf None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:07.394552 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f66ebb8-2ae5-4d91-b16f-056391c2afaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:07.418048 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 397/776] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Sep 15 07:21:07 2022] PUT /identity/v3/projects/4e9987720930473da840579654311bc2/users/a080c7ab9b684abca6828067f2572c3b/roles/6bbb643e0d8d4bc0a9b27c26edf59216 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:21:07.418901 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1a7986-f714-4159-a1e6-2482ba5b1438 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:07.420138 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-8f66ebb8-2ae5-4d91-b16f-056391c2afaf None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:21:07.421428 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-8f66ebb8-2ae5-4d91-b16f-056391c2afaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:21:07.522566 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 380/777] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:21:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 07:21:07.529642 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982ac3db-e93e-46ec-b6e8-48321ca1254b None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:07.529642 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982ac3db-e93e-46ec-b6e8-48321ca1254b None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:07.529642 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982ac3db-e93e-46ec-b6e8-48321ca1254b None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:07.529642 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 381/778] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:21:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:21:07.534041 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aafb5a0-61e7-4768-b072-3ef4762d4405 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:07.534343 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aafb5a0-61e7-4768-b072-3ef4762d4405 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:07.534598 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aafb5a0-61e7-4768-b072-3ef4762d4405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:07.566828 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-4aafb5a0-61e7-4768-b072-3ef4762d4405 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:21:07.568047 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-4aafb5a0-61e7-4768-b072-3ef4762d4405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:21:07.608091 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1a7986-f714-4159-a1e6-2482ba5b1438 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:07.650073 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:07.656942 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e58e02f-d64e-4ba9-84a6-1947d6b8854f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:07.658101 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e58e02f-d64e-4ba9-84a6-1947d6b8854f None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:07.658385 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e58e02f-d64e-4ba9-84a6-1947d6b8854f None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:07.658663 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e58e02f-d64e-4ba9-84a6-1947d6b8854f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:07.668684 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 382/779] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:21:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 07:21:07.672392 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e58e02f-d64e-4ba9-84a6-1947d6b8854f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:21:07.677265 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-7e58e02f-d64e-4ba9-84a6-1947d6b8854f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 07:21:07.677265 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:07.677265 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:07.677265 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:07.677265 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:07.677265 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:07.677265 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:07.677265 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:07.677265 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:07.677265 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:07.677265 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:07.677265 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:07.677265 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:21:07.677265 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:21:07.677265 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:21:07.677265 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:21:07.678226 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:21:07.678226 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:21:07.678226 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:21:07.678226 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:21:07.678226 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:07.678226 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:07.678226 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:07.678226 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:07.678226 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:07.678226 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:07.678226 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:07.678226 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:07.678226 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:07.678226 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:07.678226 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:07.678226 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:07.679431 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:07.679431 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:07.679431 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:07.679431 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:07.679431 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:21:07.679431 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:21:07.679431 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:21:07.679431 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:21:07.679431 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:21:07.679431 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:21:07.679431 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 07:21:07.679431 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:21:07.683186 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 398/780] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Sep 15 07:21:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:21:07.684562 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aafb5a0-61e7-4768-b072-3ef4762d4405 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:07.738050 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:07.793916 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aafb5a0-61e7-4768-b072-3ef4762d4405 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:07.829945 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d595745-a97c-4c7e-8417-dbbbf0e6b1cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:07.830561 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d595745-a97c-4c7e-8417-dbbbf0e6b1cf None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:07.830843 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d595745-a97c-4c7e-8417-dbbbf0e6b1cf None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:07.831120 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d595745-a97c-4c7e-8417-dbbbf0e6b1cf None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:07.831786 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d595745-a97c-4c7e-8417-dbbbf0e6b1cf None admin] RBAC: Adding query filter params (name=service) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:07.832092 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d595745-a97c-4c7e-8417-dbbbf0e6b1cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:07.833452 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d595745-a97c-4c7e-8417-dbbbf0e6b1cf None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:07.839551 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e58e02f-d64e-4ba9-84a6-1947d6b8854f None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:07.848658 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 383/781] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Sep 15 07:21:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:07.855629 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d595745-a97c-4c7e-8417-dbbbf0e6b1cf None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:08.000735 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d595745-a97c-4c7e-8417-dbbbf0e6b1cf None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:08.010488 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:08.018779 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e58e02f-d64e-4ba9-84a6-1947d6b8854f None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:08.039620 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9854ad67-220d-4b69-abdd-a9d283cb6733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:08.040309 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9854ad67-220d-4b69-abdd-a9d283cb6733 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:08.040803 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9854ad67-220d-4b69-abdd-a9d283cb6733 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:08.041079 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9854ad67-220d-4b69-abdd-a9d283cb6733 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:08.054451 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9854ad67-220d-4b69-abdd-a9d283cb6733 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:08.056067 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9854ad67-220d-4b69-abdd-a9d283cb6733 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:08.061455 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14045eb4-2b31-4f2e-9a07-82370adad428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:08.062119 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14045eb4-2b31-4f2e-9a07-82370adad428 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:08.062480 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14045eb4-2b31-4f2e-9a07-82370adad428 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:08.062900 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14045eb4-2b31-4f2e-9a07-82370adad428 None admin] PATH_INFO: `/v3/services/network` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:08.064065 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14045eb4-2b31-4f2e-9a07-82370adad428 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:08.065999 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14045eb4-2b31-4f2e-9a07-82370adad428 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:08.072282 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-9854ad67-220d-4b69-abdd-a9d283cb6733 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:21:08.072282 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:08.072282 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:08.072282 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:08.072282 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:08.072282 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:08.072282 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:08.072282 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:08.072282 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:08.072282 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:08.072282 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:08.072282 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:08.072282 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 07:21:08.072282 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 07:21:08.072282 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 07:21:08.072282 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 07:21:08.073223 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:08.073223 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:08.073223 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:08.073223 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:08.073223 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:08.073223 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:08.073223 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:08.073223 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:08.073223 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:08.073223 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:08.073223 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:08.073223 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:08.073223 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:08.073223 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:08.073223 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:08.073223 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:08.073223 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 07:21:08.074635 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 07:21:08.074635 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:21:08.074635 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:21:08.074635 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 384/782] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:21:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 07:21:08.075824 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9854ad67-220d-4b69-abdd-a9d283cb6733 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:08.075970 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-14045eb4-2b31-4f2e-9a07-82370adad428 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 15 07:21:08.075970 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:08.075970 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 07:21:08.075970 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 07:21:08.075970 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 07:21:08.075970 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 07:21:08.075970 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 07:21:08.075970 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 07:21:08.075970 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 15 07:21:08.075970 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:21:08.075970 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 07:21:08.075970 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:21:08.075970 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:08.075970 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:08.075970 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:08.075970 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:08.075970 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:08.077872 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:08.077872 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:08.077872 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:08.077872 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:08.077872 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:08.077872 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:08.077872 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 07:21:08.077872 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 07:21:08.077872 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 07:21:08.077872 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 07:21:08.077872 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:08.077872 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:08.077872 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:08.077872 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:08.077872 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:08.077872 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:08.080755 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:08.080755 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:08.080755 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:08.080755 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:08.080755 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:08.080755 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:08.080755 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:08.080755 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:08.080755 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:08.080755 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:08.080755 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 07:21:08.080755 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 07:21:08.080755 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 15 07:21:08.080755 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:21:08.080755 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 399/783] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 15 07:21:07 2022] GET /identity/v3/services/network => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 07:21:08.087515 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14045eb4-2b31-4f2e-9a07-82370adad428 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:08.158264 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:08.196620 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9854ad67-220d-4b69-abdd-a9d283cb6733 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:08.220985 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14045eb4-2b31-4f2e-9a07-82370adad428 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:08.240600 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12ba336-ef1a-49ff-b1f3-dce2e204556f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:08.241548 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12ba336-ef1a-49ff-b1f3-dce2e204556f None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:08.241698 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12ba336-ef1a-49ff-b1f3-dce2e204556f None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:08.241905 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12ba336-ef1a-49ff-b1f3-dce2e204556f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:08.254514 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12ba336-ef1a-49ff-b1f3-dce2e204556f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:21:08.260628 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-e12ba336-ef1a-49ff-b1f3-dce2e204556f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 07:21:08.260628 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:08.260628 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:08.260628 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:08.260628 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:08.260628 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:08.260628 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:08.260628 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:08.260628 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:08.260628 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:08.260628 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:08.260628 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:08.260628 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:21:08.260628 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:21:08.260628 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:21:08.260628 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:21:08.261572 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:21:08.261572 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:21:08.261572 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:21:08.261572 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:21:08.261572 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:08.261572 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:08.261572 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:08.261572 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:08.261572 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:08.261572 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:08.261572 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:08.261572 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:08.261572 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:08.261572 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:08.261572 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:08.261572 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:08.262727 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:08.262727 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:08.262727 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:08.262727 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:08.262727 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:21:08.262727 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:21:08.262727 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:21:08.262727 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:21:08.262727 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:21:08.262727 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:21:08.262727 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 07:21:08.262727 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:21:08.266666 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 385/784] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 15 07:21:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:21:08.267601 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12ba336-ef1a-49ff-b1f3-dce2e204556f None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:08.269388 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a862804d-666f-42fc-8f6c-aa4e121c467e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:08.270539 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a862804d-666f-42fc-8f6c-aa4e121c467e None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:08.271052 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a862804d-666f-42fc-8f6c-aa4e121c467e None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:08.272526 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a862804d-666f-42fc-8f6c-aa4e121c467e None admin] PATH_INFO: `/v3/services` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:08.273388 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a862804d-666f-42fc-8f6c-aa4e121c467e None admin] RBAC: Adding query filter params (name=network) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:08.273931 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a862804d-666f-42fc-8f6c-aa4e121c467e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:08.275508 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a862804d-666f-42fc-8f6c-aa4e121c467e None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:08.287293 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 400/785] 10.222.0.42 () {56 vars in 1112 bytes} [Thu Sep 15 07:21:08 2022] GET /identity/v3/services?name=network => generated 124 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:08.287293 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a862804d-666f-42fc-8f6c-aa4e121c467e None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:08.298460 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:08.398701 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12ba336-ef1a-49ff-b1f3-dce2e204556f None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:08.429563 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a862804d-666f-42fc-8f6c-aa4e121c467e None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:08.439658 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435d6c45-45d3-4048-abae-422751178680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:08.440320 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435d6c45-45d3-4048-abae-422751178680 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:08.440601 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435d6c45-45d3-4048-abae-422751178680 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:08.440873 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435d6c45-45d3-4048-abae-422751178680 None admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:08.441972 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435d6c45-45d3-4048-abae-422751178680 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:08.442277 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435d6c45-45d3-4048-abae-422751178680 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:08.443629 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435d6c45-45d3-4048-abae-422751178680 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:08.461774 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 386/786] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 15 07:21:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:08.461774 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435d6c45-45d3-4048-abae-422751178680 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:08.462486 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:08.482586 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac49746-2f88-441e-b655-bb7a86d64bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:08.483813 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac49746-2f88-441e-b655-bb7a86d64bb4 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:08.488712 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac49746-2f88-441e-b655-bb7a86d64bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:08.488712 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac49746-2f88-441e-b655-bb7a86d64bb4 None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:08.488712 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac49746-2f88-441e-b655-bb7a86d64bb4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:08.488712 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac49746-2f88-441e-b655-bb7a86d64bb4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:08.488712 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac49746-2f88-441e-b655-bb7a86d64bb4 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:08.500615 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 401/787] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 15 07:21:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:08.503113 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac49746-2f88-441e-b655-bb7a86d64bb4 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:08.559537 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:08.588273 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435d6c45-45d3-4048-abae-422751178680 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:08.628233 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c193f32b-957d-4bb0-a2e5-b17c4990932f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:08.629412 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c193f32b-957d-4bb0-a2e5-b17c4990932f None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:08.629907 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c193f32b-957d-4bb0-a2e5-b17c4990932f None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:08.630406 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c193f32b-957d-4bb0-a2e5-b17c4990932f None admin] PATH_INFO: `/v3/services` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:08.631244 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c193f32b-957d-4bb0-a2e5-b17c4990932f None admin] RBAC: Adding query filter params (type=network) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:08.631742 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c193f32b-957d-4bb0-a2e5-b17c4990932f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:08.633521 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c193f32b-957d-4bb0-a2e5-b17c4990932f None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:08.639311 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 387/788] 10.222.0.42 () {56 vars in 1112 bytes} [Thu Sep 15 07:21:08 2022] GET /identity/v3/services?type=network => generated 350 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:08.640303 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c193f32b-957d-4bb0-a2e5-b17c4990932f None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:08.645651 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac49746-2f88-441e-b655-bb7a86d64bb4 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:08.682750 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca777cd-b8e5-4197-be33-8dc045675df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:08.683728 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca777cd-b8e5-4197-be33-8dc045675df9 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:08.684233 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca777cd-b8e5-4197-be33-8dc045675df9 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:08.684700 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca777cd-b8e5-4197-be33-8dc045675df9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:08.692697 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca777cd-b8e5-4197-be33-8dc045675df9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:08.694457 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca777cd-b8e5-4197-be33-8dc045675df9 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:08.706891 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-7ca777cd-b8e5-4197-be33-8dc045675df9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 15 07:21:08.706891 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:08.706891 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:08.706891 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:08.706891 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:08.706891 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:08.706891 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:08.706891 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:08.706891 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:08.706891 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:08.706891 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:08.706891 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:08.706891 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 07:21:08.706891 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 07:21:08.706891 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 07:21:08.706891 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 07:21:08.707928 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:08.707928 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:08.707928 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 07:21:08.707928 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 07:21:08.707928 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 07:21:08.707928 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 07:21:08.707928 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:08.707928 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:08.707928 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:08.707928 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:08.707928 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:08.707928 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:08.707928 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:08.707928 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:08.707928 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:08.707928 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:08.707928 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:08.707928 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:08.708865 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:08.708865 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:08.708865 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 07:21:08.708865 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 07:21:08.708865 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 07:21:08.708865 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 07:21:08.708865 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 07:21:08.708865 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 07:21:08.708865 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 15 07:21:08.708865 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:21:08.711450 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 402/789] 10.222.0.42 () {56 vars in 1117 bytes} [Thu Sep 15 07:21:08 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:21:08.712553 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca777cd-b8e5-4197-be33-8dc045675df9 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:08.760575 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:08.771495 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c193f32b-957d-4bb0-a2e5-b17c4990932f None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:08.815713 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5face4-520c-4379-be15-263f9dd54d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:08.816367 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5face4-520c-4379-be15-263f9dd54d23 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:08.816653 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5face4-520c-4379-be15-263f9dd54d23 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:08.816931 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5face4-520c-4379-be15-263f9dd54d23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:08.822216 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca777cd-b8e5-4197-be33-8dc045675df9 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:08.829593 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5face4-520c-4379-be15-263f9dd54d23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:08.831574 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5face4-520c-4379-be15-263f9dd54d23 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:08.835076 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:08.850814 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-6a5face4-520c-4379-be15-263f9dd54d23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:21:08.850814 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:08.850814 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:08.850814 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:08.850814 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:08.850814 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:08.850814 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:08.850814 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:08.850814 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:08.850814 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:08.850814 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:08.850814 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:08.850814 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 07:21:08.850814 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 07:21:08.850814 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 07:21:08.850814 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 07:21:08.851844 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:08.851844 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:08.851844 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:08.851844 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:08.851844 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:08.851844 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:08.851844 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:08.851844 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:08.851844 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:08.851844 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:08.851844 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:08.851844 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:08.851844 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:08.851844 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:08.851844 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:08.851844 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:08.851844 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 07:21:08.852767 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 07:21:08.852767 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:21:08.852767 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:21:08.852767 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 388/790] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:21:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:21:08.853446 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5face4-520c-4379-be15-263f9dd54d23 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:08.863110 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ef764e-8991-4dcd-8d5a-ab43a23e1ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:08.864104 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ef764e-8991-4dcd-8d5a-ab43a23e1ebe None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:08.864626 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ef764e-8991-4dcd-8d5a-ab43a23e1ebe None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:08.865115 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ef764e-8991-4dcd-8d5a-ab43a23e1ebe None admin] PATH_INFO: `/v3/endpoints` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:08.866101 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ef764e-8991-4dcd-8d5a-ab43a23e1ebe None admin] RBAC: Adding query filter params (interface=public, service_id=a3d23dcefed94764bc225f17659bfa6b, region_id=RegionOne) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:08.866610 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ef764e-8991-4dcd-8d5a-ab43a23e1ebe None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:08.868286 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ef764e-8991-4dcd-8d5a-ab43a23e1ebe None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:08.874381 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 403/791] 10.222.0.42 () {56 vars in 1251 bytes} [Thu Sep 15 07:21:08 2022] GET /identity/v3/endpoints?service_id=a3d23dcefed94764bc225f17659bfa6b&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:08.875775 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ef764e-8991-4dcd-8d5a-ab43a23e1ebe None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:08.984556 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5face4-520c-4379-be15-263f9dd54d23 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:09.022177 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ef764e-8991-4dcd-8d5a-ab43a23e1ebe None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:09.023011 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb55e58-b1cc-4926-8c90-5438fe8ba622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:09.024222 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb55e58-b1cc-4926-8c90-5438fe8ba622 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:09.024699 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb55e58-b1cc-4926-8c90-5438fe8ba622 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:09.025182 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb55e58-b1cc-4926-8c90-5438fe8ba622 None admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:09.026207 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb55e58-b1cc-4926-8c90-5438fe8ba622 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:09.026714 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb55e58-b1cc-4926-8c90-5438fe8ba622 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:09.028332 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb55e58-b1cc-4926-8c90-5438fe8ba622 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:09.029308 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:09.050433 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 389/792] 10.222.0.42 () {56 vars in 1149 bytes} [Thu Sep 15 07:21:08 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:09.051790 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb55e58-b1cc-4926-8c90-5438fe8ba622 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:09.075046 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fa1277-a753-4bd4-a81b-203212baff9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:09.075662 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fa1277-a753-4bd4-a81b-203212baff9b None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:09.076011 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fa1277-a753-4bd4-a81b-203212baff9b None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:09.076292 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fa1277-a753-4bd4-a81b-203212baff9b None admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:09.076933 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fa1277-a753-4bd4-a81b-203212baff9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:09.077245 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fa1277-a753-4bd4-a81b-203212baff9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:09.078737 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fa1277-a753-4bd4-a81b-203212baff9b None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:09.097250 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 404/793] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 15 07:21:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:09.098131 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fa1277-a753-4bd4-a81b-203212baff9b None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:09.120723 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:09.179453 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb55e58-b1cc-4926-8c90-5438fe8ba622 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:09.219753 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059a9440-4858-4a5a-92bd-0f5e47d67dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:09.220922 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059a9440-4858-4a5a-92bd-0f5e47d67dd3 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:09.221401 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059a9440-4858-4a5a-92bd-0f5e47d67dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:09.221902 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059a9440-4858-4a5a-92bd-0f5e47d67dd3 None admin] PATH_INFO: `/v3/services` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:09.222699 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059a9440-4858-4a5a-92bd-0f5e47d67dd3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:09.224986 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059a9440-4858-4a5a-92bd-0f5e47d67dd3 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:09.234091 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 390/794] 10.222.0.42 () {56 vars in 1087 bytes} [Thu Sep 15 07:21:09 2022] GET /identity/v3/services => generated 772 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:09.235201 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fa1277-a753-4bd4-a81b-203212baff9b None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:09.236585 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059a9440-4858-4a5a-92bd-0f5e47d67dd3 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:09.311690 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d168c5d-f8ba-4276-90f9-33067dfae601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:09.313004 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d168c5d-f8ba-4276-90f9-33067dfae601 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:09.317665 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d168c5d-f8ba-4276-90f9-33067dfae601 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:09.318348 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d168c5d-f8ba-4276-90f9-33067dfae601 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:09.336151 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d168c5d-f8ba-4276-90f9-33067dfae601 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:09.339556 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d168c5d-f8ba-4276-90f9-33067dfae601 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:09.364920 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-2d168c5d-f8ba-4276-90f9-33067dfae601 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:21:09.364920 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:09.364920 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:09.364920 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:09.364920 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:09.364920 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:09.364920 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:09.364920 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:09.364920 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:09.364920 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:09.364920 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:09.364920 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:09.364920 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 07:21:09.364920 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 07:21:09.364920 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 07:21:09.364920 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 07:21:09.365935 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:09.365935 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:09.365935 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:09.365935 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:09.365935 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:09.365935 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:09.365935 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:09.365935 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:09.365935 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:09.365935 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:09.365935 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:09.365935 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:09.365935 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:09.365935 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:09.365935 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:09.365935 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:09.365935 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 07:21:09.366834 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 07:21:09.366834 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:21:09.366834 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:21:09.369119 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 405/795] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:21:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:21:09.375838 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d168c5d-f8ba-4276-90f9-33067dfae601 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:09.415688 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:09.452506 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059a9440-4858-4a5a-92bd-0f5e47d67dd3 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:09.475532 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:09.493178 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aaaf5e-2b00-4d38-980d-dfb586fe5791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:09.493811 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aaaf5e-2b00-4d38-980d-dfb586fe5791 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:09.494097 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aaaf5e-2b00-4d38-980d-dfb586fe5791 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:09.494369 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aaaf5e-2b00-4d38-980d-dfb586fe5791 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:09.503029 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9aaaf5e-2b00-4d38-980d-dfb586fe5791 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:09.505105 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9aaaf5e-2b00-4d38-980d-dfb586fe5791 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:09.521199 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-b9aaaf5e-2b00-4d38-980d-dfb586fe5791 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 15 07:21:09.521199 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:09.521199 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:09.521199 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:09.521199 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:09.521199 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:09.521199 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:09.521199 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:09.521199 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:09.521199 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:09.521199 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:09.521199 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:09.521199 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 07:21:09.521199 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 07:21:09.521199 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 07:21:09.521199 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 07:21:09.522251 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:09.522251 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:09.522251 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 07:21:09.522251 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 07:21:09.522251 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 07:21:09.522251 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 07:21:09.522251 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:09.522251 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:09.522251 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:09.522251 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:09.522251 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:09.522251 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:09.522251 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:09.522251 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:09.522251 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:09.522251 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:09.522251 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:09.522251 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:09.523199 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:09.523199 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:09.523199 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 07:21:09.523199 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 07:21:09.523199 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 07:21:09.523199 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 07:21:09.523199 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 07:21:09.523199 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 07:21:09.523199 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 15 07:21:09.523199 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:21:09.523199 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 391/796] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 15 07:21:09 2022] GET /identity/v3/users/swift => generated 83 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:21:09.530865 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aaaf5e-2b00-4d38-980d-dfb586fe5791 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:09.537739 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d168c5d-f8ba-4276-90f9-33067dfae601 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:09.589733 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883d4de6-2154-4716-b60b-04cba2a4a6c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:09.590837 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883d4de6-2154-4716-b60b-04cba2a4a6c8 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:09.591123 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883d4de6-2154-4716-b60b-04cba2a4a6c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:09.591539 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883d4de6-2154-4716-b60b-04cba2a4a6c8 None admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:09.592324 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883d4de6-2154-4716-b60b-04cba2a4a6c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:09.592638 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883d4de6-2154-4716-b60b-04cba2a4a6c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:09.594906 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883d4de6-2154-4716-b60b-04cba2a4a6c8 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:09.615005 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 406/797] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 15 07:21:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:21:09.620731 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883d4de6-2154-4716-b60b-04cba2a4a6c8 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:09.655686 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aaaf5e-2b00-4d38-980d-dfb586fe5791 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:09.719757 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a51d8c7-4e7d-4b6c-af7a-442fec0abcd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:09.720556 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a51d8c7-4e7d-4b6c-af7a-442fec0abcd8 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:09.720835 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a51d8c7-4e7d-4b6c-af7a-442fec0abcd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:09.721087 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a51d8c7-4e7d-4b6c-af7a-442fec0abcd8 None admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:09.721849 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a51d8c7-4e7d-4b6c-af7a-442fec0abcd8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:09.722151 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a51d8c7-4e7d-4b6c-af7a-442fec0abcd8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:09.723559 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a51d8c7-4e7d-4b6c-af7a-442fec0abcd8 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:09.744888 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:09.757520 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 392/798] 10.222.0.42 () {56 vars in 1135 bytes} [Thu Sep 15 07:21:09 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:09.758898 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9e6361-970c-4c7b-9943-e2b0877ded06 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:09.759182 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9e6361-970c-4c7b-9943-e2b0877ded06 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:09.759453 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9e6361-970c-4c7b-9943-e2b0877ded06 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:09.764812 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 393/799] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:21:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:21:09.764812 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9e6361-970c-4c7b-9943-e2b0877ded06 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:09.781924 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883d4de6-2154-4716-b60b-04cba2a4a6c8 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:09.805732 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:09.841523 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953bbe59-89cd-427e-b635-f45be803a302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:09.842623 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953bbe59-89cd-427e-b635-f45be803a302 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:09.843334 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953bbe59-89cd-427e-b635-f45be803a302 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:09.845034 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953bbe59-89cd-427e-b635-f45be803a302 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:09.854544 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953bbe59-89cd-427e-b635-f45be803a302 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:09.856779 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953bbe59-89cd-427e-b635-f45be803a302 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:09.873199 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-953bbe59-89cd-427e-b635-f45be803a302 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 07:21:09.873199 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:09.873199 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:09.873199 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:09.873199 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:09.873199 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:09.873199 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:09.873199 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:09.873199 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:09.873199 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:09.873199 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:09.873199 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:09.873199 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 07:21:09.873199 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 07:21:09.873199 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 07:21:09.874176 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 07:21:09.874176 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:09.874176 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:09.874176 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:09.874176 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:09.874176 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:09.874176 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:09.874176 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:09.874176 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:09.874176 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:09.874176 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:09.874176 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:09.874176 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:09.874176 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:09.874176 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:09.874176 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:09.874176 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:09.875678 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 07:21:09.875678 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 07:21:09.875678 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 07:21:09.875678 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 07:21:09.875678 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 07:21:09.875678 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 07:21:09.875678 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 07:21:09.875678 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:21:09.879081 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 407/800] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 15 07:21:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 07:21:09.881366 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c47e768-18a3-4ba8-ae65-e678e26abac6 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:09.881906 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c47e768-18a3-4ba8-ae65-e678e26abac6 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:09.882389 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c47e768-18a3-4ba8-ae65-e678e26abac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:09.887428 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9e6361-970c-4c7b-9943-e2b0877ded06 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:09.907068 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-7c47e768-18a3-4ba8-ae65-e678e26abac6 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:21:09.908406 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-7c47e768-18a3-4ba8-ae65-e678e26abac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:21:09.940981 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912e3da4-cc57-49b5-8f55-dc4cd727ee60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:09.942042 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e3da4-cc57-49b5-8f55-dc4cd727ee60 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:09.942579 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e3da4-cc57-49b5-8f55-dc4cd727ee60 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:09.943700 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e3da4-cc57-49b5-8f55-dc4cd727ee60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:09.957543 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912e3da4-cc57-49b5-8f55-dc4cd727ee60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:09.959037 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912e3da4-cc57-49b5-8f55-dc4cd727ee60 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:09.974386 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-912e3da4-cc57-49b5-8f55-dc4cd727ee60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:21:09.974386 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:09.974386 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:09.974386 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:09.974386 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:09.974386 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:09.974386 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:09.974386 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:09.974386 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:09.974386 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:09.974386 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:09.974386 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:09.974386 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 07:21:09.974386 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 07:21:09.974386 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 07:21:09.974386 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 07:21:09.975355 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:09.975355 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:09.975355 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:09.975355 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:09.975355 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:09.975355 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:09.975355 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:09.975355 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:09.975355 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:09.975355 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:09.975355 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:09.975355 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:09.975355 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:09.975355 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:09.975355 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:09.975355 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:09.975355 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 07:21:09.976251 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 07:21:09.976251 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:21:09.976251 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:21:09.976251 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 394/801] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:21:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:21:09.976251 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912e3da4-cc57-49b5-8f55-dc4cd727ee60 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:09.991375 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 408/802] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:21:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 07:21:09.992484 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49125e2b-0c53-4b22-8739-40459f59d375 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:09.992768 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49125e2b-0c53-4b22-8739-40459f59d375 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:09.993038 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49125e2b-0c53-4b22-8739-40459f59d375 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:09.998467 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 409/803] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:21:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:21:09.998467 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49125e2b-0c53-4b22-8739-40459f59d375 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:10.071687 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:10.100378 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:10.115059 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912e3da4-cc57-49b5-8f55-dc4cd727ee60 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:10.116179 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49125e2b-0c53-4b22-8739-40459f59d375 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:10.152708 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14551360-661b-45d3-87aa-b8fcfc64b011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:10.153787 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14551360-661b-45d3-87aa-b8fcfc64b011 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:10.154145 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14551360-661b-45d3-87aa-b8fcfc64b011 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:10.154493 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14551360-661b-45d3-87aa-b8fcfc64b011 None admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:10.155279 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14551360-661b-45d3-87aa-b8fcfc64b011 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:10.155752 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14551360-661b-45d3-87aa-b8fcfc64b011 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:10.157369 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14551360-661b-45d3-87aa-b8fcfc64b011 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:10.159873 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42159bea-87fd-493c-bdc6-fce6902cd14b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:10.161074 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42159bea-87fd-493c-bdc6-fce6902cd14b None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:10.161541 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42159bea-87fd-493c-bdc6-fce6902cd14b None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:10.162017 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42159bea-87fd-493c-bdc6-fce6902cd14b None admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:10.162825 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42159bea-87fd-493c-bdc6-fce6902cd14b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:10.163303 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42159bea-87fd-493c-bdc6-fce6902cd14b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:10.164825 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42159bea-87fd-493c-bdc6-fce6902cd14b None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:10.174719 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 395/804] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 15 07:21:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:10.176233 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed866e3-b36e-442d-ad2f-4e15b11e32e4 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:10.176590 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed866e3-b36e-442d-ad2f-4e15b11e32e4 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:10.176971 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed866e3-b36e-442d-ad2f-4e15b11e32e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:10.184470 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 410/805] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 15 07:21:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:10.185928 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42159bea-87fd-493c-bdc6-fce6902cd14b None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:10.209961 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-bed866e3-b36e-442d-ad2f-4e15b11e32e4 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:21:10.211171 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-bed866e3-b36e-442d-ad2f-4e15b11e32e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:21:10.291224 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 396/806] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:21:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 07:21:10.292052 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed866e3-b36e-442d-ad2f-4e15b11e32e4 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:10.318835 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42159bea-87fd-493c-bdc6-fce6902cd14b None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:10.345829 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:10.354635 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8821f9-b0a3-42bb-a0b5-81bc02ee1c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:10.355745 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8821f9-b0a3-42bb-a0b5-81bc02ee1c89 None admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:10.356293 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8821f9-b0a3-42bb-a0b5-81bc02ee1c89 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:10.356798 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8821f9-b0a3-42bb-a0b5-81bc02ee1c89 None admin] PATH_INFO: `/v3/projects/4e9987720930473da840579654311bc2/users/f9c68c4bd9074d8d9b893755e28c0807/roles/0e661b5a67664692b933e5da9fc2e488` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:10.373220 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8821f9-b0a3-42bb-a0b5-81bc02ee1c89 None admin] RBAC: Authorizing `identity:create_grant(project_id=4e9987720930473da840579654311bc2, user_id=f9c68c4bd9074d8d9b893755e28c0807, role_id=0e661b5a67664692b933e5da9fc2e488)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:10.375424 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8821f9-b0a3-42bb-a0b5-81bc02ee1c89 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:10.384810 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:10.406074 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed866e3-b36e-442d-ad2f-4e15b11e32e4 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:10.438970 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 411/807] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Sep 15 07:21:10 2022] PUT /identity/v3/projects/4e9987720930473da840579654311bc2/users/f9c68c4bd9074d8d9b893755e28c0807/roles/0e661b5a67664692b933e5da9fc2e488 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:21:10.440108 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8821f9-b0a3-42bb-a0b5-81bc02ee1c89 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:10.441186 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bf4609-ea2d-412c-a13b-93a7336493fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:10.450609 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bf4609-ea2d-412c-a13b-93a7336493fa None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:10.450939 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bf4609-ea2d-412c-a13b-93a7336493fa None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:10.451222 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bf4609-ea2d-412c-a13b-93a7336493fa None admin] PATH_INFO: `/v3/projects/service` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:10.466194 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bf4609-ea2d-412c-a13b-93a7336493fa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:10.467806 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bf4609-ea2d-412c-a13b-93a7336493fa None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:10.490740 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-25bf4609-ea2d-412c-a13b-93a7336493fa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 07:21:10.490740 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:10.490740 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:10.490740 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:10.490740 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:10.490740 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:10.490740 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:10.490740 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:10.490740 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:10.490740 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:10.490740 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:10.490740 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:10.490740 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 07:21:10.490740 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 07:21:10.490740 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 07:21:10.491729 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 07:21:10.491729 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:10.491729 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:10.491729 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:10.491729 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:10.491729 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:10.491729 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:10.491729 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:10.491729 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:10.491729 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:10.491729 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:10.491729 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:10.491729 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:10.491729 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:10.491729 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:10.491729 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:10.491729 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:10.492612 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 07:21:10.492612 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 07:21:10.492612 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 07:21:10.492612 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 07:21:10.492612 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 07:21:10.492612 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 07:21:10.492612 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 07:21:10.492612 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:21:10.492612 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 397/808] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 15 07:21:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:21:10.499869 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bf4609-ea2d-412c-a13b-93a7336493fa None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:10.584567 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8821f9-b0a3-42bb-a0b5-81bc02ee1c89 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:10.628564 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758c04b2-de6a-43df-bb4e-b9023b0b74be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:10.629624 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758c04b2-de6a-43df-bb4e-b9023b0b74be None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:10.630137 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758c04b2-de6a-43df-bb4e-b9023b0b74be None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:10.630919 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758c04b2-de6a-43df-bb4e-b9023b0b74be None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:10.631534 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758c04b2-de6a-43df-bb4e-b9023b0b74be None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:10.632883 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758c04b2-de6a-43df-bb4e-b9023b0b74be None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:10.644614 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-758c04b2-de6a-43df-bb4e-b9023b0b74be None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 15 07:21:10.644614 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:10.644614 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 07:21:10.644614 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 07:21:10.644614 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 07:21:10.644614 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 07:21:10.644614 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 07:21:10.644614 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 07:21:10.644614 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 15 07:21:10.644614 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:21:10.644614 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 07:21:10.644614 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:21:10.644614 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:10.644614 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:10.644614 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:10.644614 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:10.644614 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:10.645794 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:10.645794 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:10.645794 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:10.645794 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:10.645794 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:10.645794 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:10.645794 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 07:21:10.645794 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 07:21:10.645794 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 07:21:10.645794 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 07:21:10.645794 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:10.645794 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:10.645794 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:10.645794 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:10.645794 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:10.645794 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:10.646624 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:10.646624 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:10.646624 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:10.646624 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:10.646624 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:10.646624 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:10.646624 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:10.646624 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:10.646624 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:10.646624 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:10.646624 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 07:21:10.646624 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 07:21:10.646624 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 15 07:21:10.646624 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:21:10.646624 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 412/809] 10.222.0.42 () {56 vars in 1129 bytes} [Thu Sep 15 07:21:10 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:21:10.647426 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758c04b2-de6a-43df-bb4e-b9023b0b74be None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:10.691016 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bf4609-ea2d-412c-a13b-93a7336493fa None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:10.697055 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:10.733359 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26e7192-c88e-439e-8869-46f2d4a10209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:10.733995 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e7192-c88e-439e-8869-46f2d4a10209 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:10.734274 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e7192-c88e-439e-8869-46f2d4a10209 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:10.734609 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e7192-c88e-439e-8869-46f2d4a10209 None admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:10.735423 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26e7192-c88e-439e-8869-46f2d4a10209 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:10.735725 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26e7192-c88e-439e-8869-46f2d4a10209 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:10.737152 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26e7192-c88e-439e-8869-46f2d4a10209 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:10.745338 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:10.753144 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 398/810] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 15 07:21:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:10.757617 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26e7192-c88e-439e-8869-46f2d4a10209 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:10.780034 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758c04b2-de6a-43df-bb4e-b9023b0b74be None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:10.835767 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af23312-1095-4e00-82c2-4d996bf7c613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:10.836650 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af23312-1095-4e00-82c2-4d996bf7c613 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:10.836947 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af23312-1095-4e00-82c2-4d996bf7c613 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:10.837227 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af23312-1095-4e00-82c2-4d996bf7c613 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:10.857109 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af23312-1095-4e00-82c2-4d996bf7c613 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:21:10.867444 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-9af23312-1095-4e00-82c2-4d996bf7c613 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 07:21:10.867444 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:10.867444 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:10.867444 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:10.867444 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:10.867444 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:10.867444 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:10.867444 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:10.867444 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:10.867444 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:10.867444 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:10.867444 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:10.867444 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:21:10.867444 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:21:10.867444 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:21:10.867444 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:21:10.868446 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:21:10.868446 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:21:10.868446 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:21:10.868446 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:21:10.868446 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:10.868446 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:10.868446 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:10.868446 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:10.868446 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:10.868446 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:10.868446 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:10.868446 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:10.868446 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:10.868446 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:10.868446 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:10.868446 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:10.869269 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:10.869269 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:10.869269 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:10.869269 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:10.869269 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:21:10.869269 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:21:10.869269 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:21:10.869269 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:21:10.869269 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:21:10.869269 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:21:10.869269 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 07:21:10.869269 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:21:10.869269 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 413/811] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 15 07:21:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:21:10.869980 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af23312-1095-4e00-82c2-4d996bf7c613 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:10.879702 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26e7192-c88e-439e-8869-46f2d4a10209 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:10.916449 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552f53d3-88ed-4f61-9ab5-078741026832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:10.917310 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552f53d3-88ed-4f61-9ab5-078741026832 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:10.917605 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552f53d3-88ed-4f61-9ab5-078741026832 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:10.917882 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552f53d3-88ed-4f61-9ab5-078741026832 None admin] PATH_INFO: `/v3/services` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:10.918508 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552f53d3-88ed-4f61-9ab5-078741026832 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:10.918808 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552f53d3-88ed-4f61-9ab5-078741026832 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:10.920214 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552f53d3-88ed-4f61-9ab5-078741026832 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:10.930481 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 399/812] 10.222.0.42 () {56 vars in 1124 bytes} [Thu Sep 15 07:21:10 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:10.931341 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552f53d3-88ed-4f61-9ab5-078741026832 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:11.000511 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:11.022222 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af23312-1095-4e00-82c2-4d996bf7c613 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:11.024410 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:11.041647 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552f53d3-88ed-4f61-9ab5-078741026832 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:11.060531 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3eaa97-b059-4baa-9a41-a4d2d210f8d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:11.061501 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3eaa97-b059-4baa-9a41-a4d2d210f8d9 None admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:11.061832 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3eaa97-b059-4baa-9a41-a4d2d210f8d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:11.062112 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3eaa97-b059-4baa-9a41-a4d2d210f8d9 None admin] PATH_INFO: `/v3/projects/4e9987720930473da840579654311bc2/users/72b90d35087e4d7a80e9417082a1541f/roles/6bbb643e0d8d4bc0a9b27c26edf59216` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:11.064388 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3eaa97-b059-4baa-9a41-a4d2d210f8d9 None admin] RBAC: Authorizing `identity:create_grant(project_id=4e9987720930473da840579654311bc2, user_id=72b90d35087e4d7a80e9417082a1541f, role_id=6bbb643e0d8d4bc0a9b27c26edf59216)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:11.066144 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3eaa97-b059-4baa-9a41-a4d2d210f8d9 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:11.077214 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855c9f81-c00d-4a4b-8292-9aad2ca8a032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:11.077841 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855c9f81-c00d-4a4b-8292-9aad2ca8a032 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:11.078675 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855c9f81-c00d-4a4b-8292-9aad2ca8a032 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:11.078955 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855c9f81-c00d-4a4b-8292-9aad2ca8a032 None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:11.079613 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855c9f81-c00d-4a4b-8292-9aad2ca8a032 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:11.079913 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855c9f81-c00d-4a4b-8292-9aad2ca8a032 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:11.081320 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855c9f81-c00d-4a4b-8292-9aad2ca8a032 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:11.090145 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 400/813] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 15 07:21:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:11.090686 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855c9f81-c00d-4a4b-8292-9aad2ca8a032 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:11.145917 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 414/814] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Sep 15 07:21:10 2022] PUT /identity/v3/projects/4e9987720930473da840579654311bc2/users/72b90d35087e4d7a80e9417082a1541f/roles/6bbb643e0d8d4bc0a9b27c26edf59216 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:21:11.146994 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3eaa97-b059-4baa-9a41-a4d2d210f8d9 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:11.224694 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855c9f81-c00d-4a4b-8292-9aad2ca8a032 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:11.258744 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:11.274085 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6e9959-ff68-4f20-b570-11bc420c1647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:11.275084 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6e9959-ff68-4f20-b570-11bc420c1647 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:11.275572 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6e9959-ff68-4f20-b570-11bc420c1647 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:11.276078 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6e9959-ff68-4f20-b570-11bc420c1647 None admin] PATH_INFO: `/v3/services` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:11.277051 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6e9959-ff68-4f20-b570-11bc420c1647 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:11.277655 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6e9959-ff68-4f20-b570-11bc420c1647 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:11.279290 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6e9959-ff68-4f20-b570-11bc420c1647 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:11.286489 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 401/815] 10.222.0.42 () {56 vars in 1124 bytes} [Thu Sep 15 07:21:11 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:11.295760 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6e9959-ff68-4f20-b570-11bc420c1647 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:11.436310 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6e9959-ff68-4f20-b570-11bc420c1647 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:11.499218 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5dc829-45fc-496a-af72-0390ff6b1ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:11.501602 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5dc829-45fc-496a-af72-0390ff6b1ea1 None admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:11.502966 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5dc829-45fc-496a-af72-0390ff6b1ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:11.503520 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5dc829-45fc-496a-af72-0390ff6b1ea1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:11.504531 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5dc829-45fc-496a-af72-0390ff6b1ea1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:11.506276 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5dc829-45fc-496a-af72-0390ff6b1ea1 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:11.539515 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3eaa97-b059-4baa-9a41-a4d2d210f8d9 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:11.581914 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f88f42-e3af-4af3-a2cd-9ca5a0204035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:11.582302 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f88f42-e3af-4af3-a2cd-9ca5a0204035 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:11.582520 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f88f42-e3af-4af3-a2cd-9ca5a0204035 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:11.582792 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f88f42-e3af-4af3-a2cd-9ca5a0204035 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:11.595304 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f88f42-e3af-4af3-a2cd-9ca5a0204035 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:11.596445 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f88f42-e3af-4af3-a2cd-9ca5a0204035 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:11.612597 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-01f88f42-e3af-4af3-a2cd-9ca5a0204035 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:21:11.612597 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:11.612597 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:11.612597 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:11.612597 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:11.612597 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:11.612597 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:11.612597 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:11.612597 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:11.612597 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:11.612597 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:11.612597 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:11.612597 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 07:21:11.612597 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 07:21:11.612597 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 07:21:11.612597 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 07:21:11.613611 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:11.613611 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:11.613611 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:11.613611 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:11.613611 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:11.613611 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:11.613611 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:11.613611 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:11.613611 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:11.613611 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:11.613611 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:11.613611 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:11.613611 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:11.613611 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:11.613611 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:11.613611 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:11.613611 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 07:21:11.614614 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 07:21:11.614614 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:21:11.614614 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:21:11.614614 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 415/816] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:21:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 467 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:21:11.622626 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f88f42-e3af-4af3-a2cd-9ca5a0204035 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:11.660394 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:11.676561 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 402/817] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:21:11 2022] POST /identity/v3/endpoints => generated 346 bytes in 381 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:21:11.816524 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f88f42-e3af-4af3-a2cd-9ca5a0204035 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:11.870757 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93998c58-4a26-478a-82d3-cd882ff9f36e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:11.871374 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93998c58-4a26-478a-82d3-cd882ff9f36e None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:11.871653 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93998c58-4a26-478a-82d3-cd882ff9f36e None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:11.871928 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93998c58-4a26-478a-82d3-cd882ff9f36e None admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:11.872565 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93998c58-4a26-478a-82d3-cd882ff9f36e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:11.872867 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93998c58-4a26-478a-82d3-cd882ff9f36e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:11.874261 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93998c58-4a26-478a-82d3-cd882ff9f36e None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:11.887325 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 416/818] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 15 07:21:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:11.893602 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5dc829-45fc-496a-af72-0390ff6b1ea1 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:11.921428 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba43134-2646-4296-9387-b114d3d5115e None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:11.921726 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba43134-2646-4296-9387-b114d3d5115e None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:11.922084 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba43134-2646-4296-9387-b114d3d5115e None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:11.924565 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 417/819] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:21:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 07:21:11.931141 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12084831-efa6-4542-b257-dcf73e192526 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:11.931699 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12084831-efa6-4542-b257-dcf73e192526 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:11.932154 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12084831-efa6-4542-b257-dcf73e192526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:11.956491 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-12084831-efa6-4542-b257-dcf73e192526 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:21:11.958161 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-12084831-efa6-4542-b257-dcf73e192526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:21:12.013320 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:12.067104 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:12.072044 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 418/820] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:21:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 07:21:12.088670 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5dc829-45fc-496a-af72-0390ff6b1ea1 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:12.128323 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d03d42c-22af-4c74-8ff9-4ce8e0e0a10c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:12.129327 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d03d42c-22af-4c74-8ff9-4ce8e0e0a10c None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:12.129789 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d03d42c-22af-4c74-8ff9-4ce8e0e0a10c None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:12.130209 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d03d42c-22af-4c74-8ff9-4ce8e0e0a10c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:12.142916 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d03d42c-22af-4c74-8ff9-4ce8e0e0a10c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:12.144374 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d03d42c-22af-4c74-8ff9-4ce8e0e0a10c None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:12.160333 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-6d03d42c-22af-4c74-8ff9-4ce8e0e0a10c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 15 07:21:12.160333 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:12.160333 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:12.160333 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:12.160333 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:12.160333 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:12.160333 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:12.160333 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:12.160333 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:12.160333 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:12.160333 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:12.160333 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:12.160333 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 07:21:12.160333 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 07:21:12.160333 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 07:21:12.160333 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 07:21:12.161948 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:12.161948 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:12.161948 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 07:21:12.161948 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 07:21:12.161948 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 07:21:12.161948 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 07:21:12.161948 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:12.161948 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:12.161948 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:12.161948 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:12.161948 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:12.161948 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:12.161948 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:12.161948 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:12.161948 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:12.161948 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:12.161948 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:12.161948 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:12.163042 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:12.163042 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:12.163042 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 07:21:12.163042 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 07:21:12.163042 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 07:21:12.163042 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 07:21:12.163042 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 07:21:12.163042 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 07:21:12.163042 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 15 07:21:12.163042 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:21:12.163042 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 403/821] 10.222.0.42 () {56 vars in 1093 bytes} [Thu Sep 15 07:21:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:21:12.170752 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12084831-efa6-4542-b257-dcf73e192526 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:12.314214 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12084831-efa6-4542-b257-dcf73e192526 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:12.363084 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1715194-f0e9-4e93-8c94-6b89d71e5c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:12.363639 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1715194-f0e9-4e93-8c94-6b89d71e5c71 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:12.363922 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1715194-f0e9-4e93-8c94-6b89d71e5c71 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:12.364232 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1715194-f0e9-4e93-8c94-6b89d71e5c71 None admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:12.373851 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1715194-f0e9-4e93-8c94-6b89d71e5c71 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:12.374063 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1715194-f0e9-4e93-8c94-6b89d71e5c71 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:12.375827 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1715194-f0e9-4e93-8c94-6b89d71e5c71 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:12.400354 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 419/822] 10.222.0.42 () {56 vars in 1133 bytes} [Thu Sep 15 07:21:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:12.406856 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d03d42c-22af-4c74-8ff9-4ce8e0e0a10c None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:12.495351 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1715194-f0e9-4e93-8c94-6b89d71e5c71 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:12.535345 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d03d42c-22af-4c74-8ff9-4ce8e0e0a10c None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:12.538412 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:12.573794 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:12.576442 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5ea41c-c097-4488-9b39-dcb2c5750eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:12.577054 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5ea41c-c097-4488-9b39-dcb2c5750eee None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:12.577331 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5ea41c-c097-4488-9b39-dcb2c5750eee None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:12.577618 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5ea41c-c097-4488-9b39-dcb2c5750eee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:12.622313 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1715194-f0e9-4e93-8c94-6b89d71e5c71 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:12.656002 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5ea41c-c097-4488-9b39-dcb2c5750eee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:12.657786 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5ea41c-c097-4488-9b39-dcb2c5750eee None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:12.664679 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efb3841-aad9-4a29-ac3d-fc8ad8b541e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:12.665670 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efb3841-aad9-4a29-ac3d-fc8ad8b541e8 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:12.666153 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efb3841-aad9-4a29-ac3d-fc8ad8b541e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:12.666638 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efb3841-aad9-4a29-ac3d-fc8ad8b541e8 None admin] PATH_INFO: `/v3/services/network` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:12.667417 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efb3841-aad9-4a29-ac3d-fc8ad8b541e8 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:12.669091 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efb3841-aad9-4a29-ac3d-fc8ad8b541e8 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:12.679944 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-7efb3841-aad9-4a29-ac3d-fc8ad8b541e8 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 15 07:21:12.679944 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:12.679944 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 07:21:12.679944 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 07:21:12.679944 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 07:21:12.679944 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 07:21:12.679944 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 07:21:12.679944 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 07:21:12.679944 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 15 07:21:12.679944 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:21:12.679944 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 07:21:12.679944 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:21:12.679944 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:12.679944 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:12.679944 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:12.679944 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:12.679944 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:12.681011 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:12.681011 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:12.681011 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:12.681011 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:12.681011 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:12.681011 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:12.681011 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 07:21:12.681011 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 07:21:12.681011 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 07:21:12.681011 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 07:21:12.681011 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:12.681011 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:12.681011 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:12.681011 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:12.681011 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:12.681011 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:12.681874 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:12.681874 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:12.681874 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:12.681874 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:12.681874 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:12.681874 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:12.681874 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:12.681874 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:12.681874 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:12.681874 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:12.681874 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 07:21:12.681874 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 07:21:12.681874 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 15 07:21:12.681874 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:21:12.684457 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 420/823] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 15 07:21:12 2022] GET /identity/v3/services/network => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 07:21:12.691028 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efb3841-aad9-4a29-ac3d-fc8ad8b541e8 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:12.772880 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 404/824] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 15 07:21:12 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 07:21:12.777932 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5ea41c-c097-4488-9b39-dcb2c5750eee None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:12.785929 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:12.848833 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efb3841-aad9-4a29-ac3d-fc8ad8b541e8 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:12.877018 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:12.892694 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537810f2-9aee-4c75-a3cb-7166989e23ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:12.893313 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537810f2-9aee-4c75-a3cb-7166989e23ca None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:12.893601 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537810f2-9aee-4c75-a3cb-7166989e23ca None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:12.893886 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537810f2-9aee-4c75-a3cb-7166989e23ca None admin] PATH_INFO: `/v3/services` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:12.895731 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537810f2-9aee-4c75-a3cb-7166989e23ca None admin] RBAC: Adding query filter params (name=network) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:12.895731 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537810f2-9aee-4c75-a3cb-7166989e23ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:12.896161 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537810f2-9aee-4c75-a3cb-7166989e23ca None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:12.905778 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 421/825] 10.222.0.42 () {56 vars in 1112 bytes} [Thu Sep 15 07:21:12 2022] GET /identity/v3/services?name=network => generated 124 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:21:12.910326 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537810f2-9aee-4c75-a3cb-7166989e23ca None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:12.972083 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5ea41c-c097-4488-9b39-dcb2c5750eee None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:13.045860 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eddda24-4fa3-49b6-ab59-b9536c30a71f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:13.046504 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eddda24-4fa3-49b6-ab59-b9536c30a71f None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:13.046783 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eddda24-4fa3-49b6-ab59-b9536c30a71f None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:13.047123 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eddda24-4fa3-49b6-ab59-b9536c30a71f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:13.064847 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eddda24-4fa3-49b6-ab59-b9536c30a71f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:13.066424 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eddda24-4fa3-49b6-ab59-b9536c30a71f None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:13.091239 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-1eddda24-4fa3-49b6-ab59-b9536c30a71f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:21:13.091239 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:13.091239 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:13.091239 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:13.091239 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:13.091239 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:13.091239 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:13.091239 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:13.091239 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:13.091239 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:13.091239 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:13.091239 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:13.091239 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 07:21:13.091239 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 07:21:13.091239 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 07:21:13.091239 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 07:21:13.092227 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:13.092227 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:13.092227 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:13.092227 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:13.092227 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:13.092227 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:13.092227 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:13.092227 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:13.092227 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:13.092227 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:13.092227 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:13.092227 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:13.092227 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:13.092227 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:13.092227 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:13.092227 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:13.092227 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 07:21:13.093087 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 07:21:13.093087 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:21:13.093087 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:21:13.093087 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 405/826] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:21:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 07:21:13.098518 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eddda24-4fa3-49b6-ab59-b9536c30a71f None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:13.112287 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537810f2-9aee-4c75-a3cb-7166989e23ca None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:13.152462 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cff3eb9-e627-48fd-85fe-4c9985d08eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:13.152990 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cff3eb9-e627-48fd-85fe-4c9985d08eee None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:13.153266 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cff3eb9-e627-48fd-85fe-4c9985d08eee None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:13.153538 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cff3eb9-e627-48fd-85fe-4c9985d08eee None admin] PATH_INFO: `/v3/services` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:13.154195 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cff3eb9-e627-48fd-85fe-4c9985d08eee None admin] RBAC: Adding query filter params (type=network) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:13.154500 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cff3eb9-e627-48fd-85fe-4c9985d08eee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:13.155868 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cff3eb9-e627-48fd-85fe-4c9985d08eee None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:13.161631 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 422/827] 10.222.0.42 () {56 vars in 1112 bytes} [Thu Sep 15 07:21:12 2022] GET /identity/v3/services?type=network => generated 350 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:13.166560 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05627b36-0995-429a-8fb4-9a987197e499 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:13.166560 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05627b36-0995-429a-8fb4-9a987197e499 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:13.166560 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05627b36-0995-429a-8fb4-9a987197e499 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:13.168938 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 423/828] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:21:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:21:13.169711 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05627b36-0995-429a-8fb4-9a987197e499 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:13.185033 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:13.228516 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:13.238982 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eddda24-4fa3-49b6-ab59-b9536c30a71f None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:13.279791 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41052d9-9537-49c2-83e7-e6e37b30b2cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:13.280971 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41052d9-9537-49c2-83e7-e6e37b30b2cd None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:13.281500 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41052d9-9537-49c2-83e7-e6e37b30b2cd None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:13.282263 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41052d9-9537-49c2-83e7-e6e37b30b2cd None admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:13.283103 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41052d9-9537-49c2-83e7-e6e37b30b2cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:13.283589 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41052d9-9537-49c2-83e7-e6e37b30b2cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:13.285320 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41052d9-9537-49c2-83e7-e6e37b30b2cd None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:13.294962 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05627b36-0995-429a-8fb4-9a987197e499 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:13.305743 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 406/829] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 15 07:21:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:13.306305 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40377da2-ab3a-4028-8137-38d01347bd17 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:13.306414 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40377da2-ab3a-4028-8137-38d01347bd17 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:13.306635 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40377da2-ab3a-4028-8137-38d01347bd17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:13.327058 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-40377da2-ab3a-4028-8137-38d01347bd17 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:21:13.328807 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-40377da2-ab3a-4028-8137-38d01347bd17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:21:13.338855 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb4d54b-3197-42a3-9a24-ef9c4c72fcba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:13.339718 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb4d54b-3197-42a3-9a24-ef9c4c72fcba None admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:13.340224 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb4d54b-3197-42a3-9a24-ef9c4c72fcba None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:13.340697 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb4d54b-3197-42a3-9a24-ef9c4c72fcba None admin] PATH_INFO: `/v3/endpoints` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:13.341561 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb4d54b-3197-42a3-9a24-ef9c4c72fcba None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:13.343163 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb4d54b-3197-42a3-9a24-ef9c4c72fcba None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:13.400115 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 407/830] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:21:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 07:21:13.401486 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40377da2-ab3a-4028-8137-38d01347bd17 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:13.446898 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 424/831] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:21:13 2022] POST /identity/v3/endpoints => generated 337 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:21:13.482048 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:13.566309 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40377da2-ab3a-4028-8137-38d01347bd17 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:13.620555 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fc1dd0-a899-435d-a499-c274ef67c7ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:13.621610 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fc1dd0-a899-435d-a499-c274ef67c7ea None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:13.622196 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fc1dd0-a899-435d-a499-c274ef67c7ea None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:13.622699 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fc1dd0-a899-435d-a499-c274ef67c7ea None admin] PATH_INFO: `/v3/projects/service` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:13.630921 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fc1dd0-a899-435d-a499-c274ef67c7ea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:13.632602 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fc1dd0-a899-435d-a499-c274ef67c7ea None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:13.645740 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-76fc1dd0-a899-435d-a499-c274ef67c7ea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 07:21:13.645740 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:13.645740 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:13.645740 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:13.645740 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:13.645740 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:13.645740 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:13.645740 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:13.645740 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:13.645740 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:13.645740 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:13.645740 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:13.645740 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 07:21:13.645740 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 07:21:13.645740 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 07:21:13.646703 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 07:21:13.646703 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:13.646703 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:13.646703 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:13.646703 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:13.646703 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:13.646703 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:13.646703 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:13.646703 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:13.646703 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:13.646703 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:13.646703 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:13.646703 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:13.646703 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:13.646703 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:13.646703 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:13.646703 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:13.647606 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 07:21:13.647606 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 07:21:13.647606 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 07:21:13.647606 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 07:21:13.647606 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 07:21:13.647606 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 07:21:13.647606 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 07:21:13.647606 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:21:13.650038 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 408/832] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 15 07:21:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:21:13.655989 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb4d54b-3197-42a3-9a24-ef9c4c72fcba None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:13.694171 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fc1dd0-a899-435d-a499-c274ef67c7ea None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:13.848497 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb4d54b-3197-42a3-9a24-ef9c4c72fcba None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:13.858475 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:13.880543 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fc1dd0-a899-435d-a499-c274ef67c7ea None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:13.887335 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:13.902346 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e3c060-9a4f-494d-afc5-b61b37e7a27d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:13.902971 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e3c060-9a4f-494d-afc5-b61b37e7a27d None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:13.903249 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e3c060-9a4f-494d-afc5-b61b37e7a27d None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:13.903519 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e3c060-9a4f-494d-afc5-b61b37e7a27d None admin] PATH_INFO: `/v3/projects` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:13.906408 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e3c060-9a4f-494d-afc5-b61b37e7a27d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:13.906746 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e3c060-9a4f-494d-afc5-b61b37e7a27d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:13.908337 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e3c060-9a4f-494d-afc5-b61b37e7a27d None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:13.923680 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cc5703-c7b8-412e-9ff6-0124bc06dacf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:13.924668 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 425/833] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 15 07:21:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:13.925494 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cc5703-c7b8-412e-9ff6-0124bc06dacf None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:13.926063 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cc5703-c7b8-412e-9ff6-0124bc06dacf None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:13.926537 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cc5703-c7b8-412e-9ff6-0124bc06dacf None admin] PATH_INFO: `/v3/roles/service` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:13.932571 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e3c060-9a4f-494d-afc5-b61b37e7a27d None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:13.948256 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cc5703-c7b8-412e-9ff6-0124bc06dacf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:21:13.953998 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-b4cc5703-c7b8-412e-9ff6-0124bc06dacf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 07:21:13.953998 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:13.953998 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:13.953998 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:13.953998 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:13.953998 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:13.953998 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:13.953998 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:13.953998 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:13.953998 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:13.953998 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:13.953998 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:13.953998 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:21:13.953998 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:21:13.953998 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:21:13.953998 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:21:13.955155 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:21:13.955155 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:21:13.955155 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:21:13.955155 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:21:13.955155 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:13.955155 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:13.955155 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:13.955155 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:13.955155 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:13.955155 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:13.955155 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:13.955155 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:13.955155 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:13.955155 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:13.955155 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:13.955155 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:13.956033 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:13.956033 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:13.956033 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:13.956033 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:13.956033 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:21:13.956033 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:21:13.956033 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:21:13.956033 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:21:13.956033 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:21:13.956033 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:21:13.956033 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 07:21:13.956033 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:21:13.958770 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 409/834] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Sep 15 07:21:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 07:21:13.969646 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cc5703-c7b8-412e-9ff6-0124bc06dacf None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:14.068342 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e3c060-9a4f-494d-afc5-b61b37e7a27d None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:14.107468 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222244e8-00ca-467b-b3a8-0f15e0b8fbd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:14.107995 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222244e8-00ca-467b-b3a8-0f15e0b8fbd1 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:14.108269 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222244e8-00ca-467b-b3a8-0f15e0b8fbd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:14.108536 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222244e8-00ca-467b-b3a8-0f15e0b8fbd1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:14.109245 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222244e8-00ca-467b-b3a8-0f15e0b8fbd1 None admin] RBAC: Adding query filter params (role.id=6bbb643e0d8d4bc0a9b27c26edf59216, scope.project.id=4e9987720930473da840579654311bc2, user.id=a080c7ab9b684abca6828067f2572c3b) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:14.110112 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222244e8-00ca-467b-b3a8-0f15e0b8fbd1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:14.111638 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222244e8-00ca-467b-b3a8-0f15e0b8fbd1 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:14.116931 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cc5703-c7b8-412e-9ff6-0124bc06dacf None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:14.118046 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 426/835] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 15 07:21:13 2022] GET /identity/v3/role_assignments?user.id=a080c7ab9b684abca6828067f2572c3b&scope.project.id=4e9987720930473da840579654311bc2&role.id=6bbb643e0d8d4bc0a9b27c26edf59216 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:21:14.145620 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bd77bf-daa2-4555-8805-b2117837eb65 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:14.145620 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bd77bf-daa2-4555-8805-b2117837eb65 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:14.145620 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bd77bf-daa2-4555-8805-b2117837eb65 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:14.147006 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 427/836] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:21:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 07:21:14.152904 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be19993-81c1-4683-b53d-ca6cf4be7570 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:14.152904 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be19993-81c1-4683-b53d-ca6cf4be7570 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:14.153200 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be19993-81c1-4683-b53d-ca6cf4be7570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:14.165459 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748f1b96-20d2-445a-b453-194472fb90b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:14.166546 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748f1b96-20d2-445a-b453-194472fb90b1 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:14.167026 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748f1b96-20d2-445a-b453-194472fb90b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:14.167622 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748f1b96-20d2-445a-b453-194472fb90b1 None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:14.168513 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748f1b96-20d2-445a-b453-194472fb90b1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:14.169037 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748f1b96-20d2-445a-b453-194472fb90b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:14.170666 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748f1b96-20d2-445a-b453-194472fb90b1 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:14.179345 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-9be19993-81c1-4683-b53d-ca6cf4be7570 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:21:14.180579 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-9be19993-81c1-4683-b53d-ca6cf4be7570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:21:14.182031 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 410/837] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Sep 15 07:21:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:14.188894 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748f1b96-20d2-445a-b453-194472fb90b1 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:14.218114 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:14.221625 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:14.281976 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 428/838] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:21:14 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 07:21:14.316301 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748f1b96-20d2-445a-b453-194472fb90b1 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:14.353387 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8d8fb3-60ff-4035-a857-198a94ce63c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:14.354049 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8d8fb3-60ff-4035-a857-198a94ce63c3 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:14.354327 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8d8fb3-60ff-4035-a857-198a94ce63c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:14.354597 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8d8fb3-60ff-4035-a857-198a94ce63c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:14.365476 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8d8fb3-60ff-4035-a857-198a94ce63c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:14.366865 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8d8fb3-60ff-4035-a857-198a94ce63c3 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:14.383617 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-2c8d8fb3-60ff-4035-a857-198a94ce63c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:21:14.383617 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:14.383617 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:14.383617 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:14.383617 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:14.383617 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:14.383617 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:14.383617 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:14.383617 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:14.383617 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:14.383617 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:14.383617 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:14.383617 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 07:21:14.383617 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 07:21:14.383617 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 07:21:14.383617 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 07:21:14.384578 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:14.384578 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:14.384578 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:14.384578 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:14.384578 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:14.384578 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:14.384578 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:14.384578 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:14.384578 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:14.384578 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:14.384578 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:14.384578 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:14.384578 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:14.384578 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:14.384578 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:14.384578 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:14.384578 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 07:21:14.385449 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 07:21:14.385449 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:21:14.385449 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:21:14.385449 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 411/839] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:21:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:21:14.390584 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be19993-81c1-4683-b53d-ca6cf4be7570 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:14.507691 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be19993-81c1-4683-b53d-ca6cf4be7570 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:14.543700 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d2a75e-38dd-4239-9648-85940e83aeec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:14.544667 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d2a75e-38dd-4239-9648-85940e83aeec None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:14.545168 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d2a75e-38dd-4239-9648-85940e83aeec None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:14.545653 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d2a75e-38dd-4239-9648-85940e83aeec None admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:14.546480 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d2a75e-38dd-4239-9648-85940e83aeec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:14.547542 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d2a75e-38dd-4239-9648-85940e83aeec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:14.549321 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d2a75e-38dd-4239-9648-85940e83aeec None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:14.556392 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8d8fb3-60ff-4035-a857-198a94ce63c3 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:14.564957 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 429/840] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 15 07:21:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:14.569983 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d2a75e-38dd-4239-9648-85940e83aeec None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:14.595126 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:14.649932 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:14.674333 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8d8fb3-60ff-4035-a857-198a94ce63c3 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:14.679575 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d2a75e-38dd-4239-9648-85940e83aeec None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:14.710350 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d9eae9-a6a4-4523-b7d1-11820b74cc9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:14.710983 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d9eae9-a6a4-4523-b7d1-11820b74cc9b None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:14.711257 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d9eae9-a6a4-4523-b7d1-11820b74cc9b None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:14.711527 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d9eae9-a6a4-4523-b7d1-11820b74cc9b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:14.716536 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64dc541-3019-4f48-98fd-b32da648dc97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:14.717377 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64dc541-3019-4f48-98fd-b32da648dc97 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:14.717864 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64dc541-3019-4f48-98fd-b32da648dc97 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:14.718327 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64dc541-3019-4f48-98fd-b32da648dc97 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:14.722574 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d9eae9-a6a4-4523-b7d1-11820b74cc9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:21:14.727141 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64dc541-3019-4f48-98fd-b32da648dc97 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:14.727776 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-b0d9eae9-a6a4-4523-b7d1-11820b74cc9b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 07:21:14.727776 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:14.727776 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:14.727776 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:14.727776 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:14.727776 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:14.727776 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:14.727776 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:14.727776 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:14.727776 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:14.727776 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:14.727776 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:14.727776 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:21:14.727776 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:21:14.727776 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:21:14.727776 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:21:14.728783 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:21:14.728783 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:21:14.728783 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:21:14.728783 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:21:14.728783 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:14.728783 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:14.728783 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:14.728783 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:14.728783 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:14.728783 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:14.728783 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:14.728783 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:14.728783 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:14.728783 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:14.728783 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:14.728783 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:14.730458 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:14.730458 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:14.730458 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:14.730458 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:14.730458 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:21:14.730458 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:21:14.730458 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:21:14.730458 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:21:14.730458 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:21:14.730458 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:21:14.730458 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 07:21:14.730458 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:21:14.731094 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64dc541-3019-4f48-98fd-b32da648dc97 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:14.731165 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 412/841] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 15 07:21:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 07:21:14.731165 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc142d5b-6bb0-4714-95fa-4f9748aebb87 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:14.731165 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc142d5b-6bb0-4714-95fa-4f9748aebb87 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:14.731165 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc142d5b-6bb0-4714-95fa-4f9748aebb87 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:14.732413 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 413/842] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:21:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:21:14.737387 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc142d5b-6bb0-4714-95fa-4f9748aebb87 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:14.744624 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-c64dc541-3019-4f48-98fd-b32da648dc97 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 15 07:21:14.744624 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:14.744624 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:14.744624 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:14.744624 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:14.744624 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:14.744624 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:14.744624 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:14.744624 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:14.744624 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:14.744624 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:14.744624 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:14.744624 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 07:21:14.744624 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 07:21:14.744624 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 07:21:14.744624 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 07:21:14.745541 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:14.745541 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:14.745541 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 07:21:14.745541 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 07:21:14.745541 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 07:21:14.745541 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 07:21:14.745541 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:14.745541 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:14.745541 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:14.745541 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:14.745541 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:14.745541 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:14.745541 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:14.745541 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:14.745541 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:14.745541 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:14.745541 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:14.745541 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:14.746500 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:14.746500 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:14.746500 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 07:21:14.746500 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 07:21:14.746500 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 07:21:14.746500 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 07:21:14.746500 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 07:21:14.746500 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 07:21:14.746500 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 15 07:21:14.746500 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:21:14.749945 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 430/843] 10.222.0.42 () {56 vars in 1117 bytes} [Thu Sep 15 07:21:14 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:21:14.751262 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c79953-55b1-4c91-a844-332c4f5a6e07 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:14.751534 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c79953-55b1-4c91-a844-332c4f5a6e07 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:14.751801 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c79953-55b1-4c91-a844-332c4f5a6e07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:14.771033 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-04c79953-55b1-4c91-a844-332c4f5a6e07 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:21:14.772214 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-04c79953-55b1-4c91-a844-332c4f5a6e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:21:14.849603 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 431/844] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:21:14 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 07:21:14.849603 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c79953-55b1-4c91-a844-332c4f5a6e07 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:14.858093 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc142d5b-6bb0-4714-95fa-4f9748aebb87 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:14.862570 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:14.896669 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f624de-1d04-405c-a939-2070ddaac8c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:14.897276 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f624de-1d04-405c-a939-2070ddaac8c9 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:14.897555 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f624de-1d04-405c-a939-2070ddaac8c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:14.897852 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f624de-1d04-405c-a939-2070ddaac8c9 None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:14.898510 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f624de-1d04-405c-a939-2070ddaac8c9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:14.898815 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f624de-1d04-405c-a939-2070ddaac8c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:14.900226 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f624de-1d04-405c-a939-2070ddaac8c9 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:14.902855 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:14.912396 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 414/845] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 15 07:21:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:14.918524 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f624de-1d04-405c-a939-2070ddaac8c9 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:14.994541 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c79953-55b1-4c91-a844-332c4f5a6e07 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:15.033607 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f624de-1d04-405c-a939-2070ddaac8c9 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:15.040651 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6cd9a9-c726-4022-b602-9904a70ed5d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:15.041314 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6cd9a9-c726-4022-b602-9904a70ed5d4 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:15.041624 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6cd9a9-c726-4022-b602-9904a70ed5d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:15.042431 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6cd9a9-c726-4022-b602-9904a70ed5d4 None admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:15.043164 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6cd9a9-c726-4022-b602-9904a70ed5d4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:15.043472 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6cd9a9-c726-4022-b602-9904a70ed5d4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:15.045834 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6cd9a9-c726-4022-b602-9904a70ed5d4 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:15.071744 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 432/846] 10.222.0.42 () {56 vars in 1149 bytes} [Thu Sep 15 07:21:14 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:15.071923 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e29f97-200d-4815-8931-fba6aaccb650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:15.072441 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e29f97-200d-4815-8931-fba6aaccb650 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:15.072726 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e29f97-200d-4815-8931-fba6aaccb650 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:15.072993 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e29f97-200d-4815-8931-fba6aaccb650 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:15.078243 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6cd9a9-c726-4022-b602-9904a70ed5d4 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:15.091607 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e29f97-200d-4815-8931-fba6aaccb650 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:15.093040 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e29f97-200d-4815-8931-fba6aaccb650 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:15.111290 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-65e29f97-200d-4815-8931-fba6aaccb650 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:21:15.111290 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:15.111290 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:15.111290 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:15.111290 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:15.111290 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:15.111290 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:15.111290 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:15.111290 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:15.111290 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:15.111290 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:15.111290 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:15.111290 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 07:21:15.111290 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 07:21:15.111290 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 07:21:15.111290 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 07:21:15.112295 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:15.112295 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:15.112295 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:15.112295 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:15.112295 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:15.112295 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:15.112295 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:15.112295 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:15.112295 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:15.112295 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:15.112295 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:15.112295 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:15.112295 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:15.112295 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:15.112295 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:15.112295 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:15.112295 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 07:21:15.113202 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 07:21:15.113202 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:21:15.113202 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:21:15.113202 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 415/847] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:21:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:21:15.122533 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e29f97-200d-4815-8931-fba6aaccb650 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:15.159826 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:15.181922 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:15.195627 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6cd9a9-c726-4022-b602-9904a70ed5d4 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:15.234442 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd723f7-612e-4531-90e3-b78cde34657c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:15.235080 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd723f7-612e-4531-90e3-b78cde34657c None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:15.235338 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd723f7-612e-4531-90e3-b78cde34657c None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:15.235607 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd723f7-612e-4531-90e3-b78cde34657c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:15.235928 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e29f97-200d-4815-8931-fba6aaccb650 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:15.271277 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698ef9ce-d337-4ebe-b296-dc3c8c344f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:15.271879 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698ef9ce-d337-4ebe-b296-dc3c8c344f90 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:15.272161 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698ef9ce-d337-4ebe-b296-dc3c8c344f90 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:15.272436 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698ef9ce-d337-4ebe-b296-dc3c8c344f90 None admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:15.273077 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698ef9ce-d337-4ebe-b296-dc3c8c344f90 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:15.273383 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698ef9ce-d337-4ebe-b296-dc3c8c344f90 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:15.274770 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698ef9ce-d337-4ebe-b296-dc3c8c344f90 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:15.286862 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 416/848] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 15 07:21:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:15.287514 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698ef9ce-d337-4ebe-b296-dc3c8c344f90 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:15.324649 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd723f7-612e-4531-90e3-b78cde34657c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:15.326303 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd723f7-612e-4531-90e3-b78cde34657c None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:15.395089 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:15.397097 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698ef9ce-d337-4ebe-b296-dc3c8c344f90 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:15.410953 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:15.430016 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 433/849] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 15 07:21:15 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 15 07:21:15.431166 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd723f7-612e-4531-90e3-b78cde34657c None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:15.441499 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a933c62a-dbbe-4382-8d18-4353e7ca1ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:15.442584 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a933c62a-dbbe-4382-8d18-4353e7ca1ea4 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:15.442871 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a933c62a-dbbe-4382-8d18-4353e7ca1ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:15.443148 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a933c62a-dbbe-4382-8d18-4353e7ca1ea4 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:15.444102 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a933c62a-dbbe-4382-8d18-4353e7ca1ea4 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:15.445831 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a933c62a-dbbe-4382-8d18-4353e7ca1ea4 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:15.456865 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-a933c62a-dbbe-4382-8d18-4353e7ca1ea4 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 15 07:21:15.456865 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:15.456865 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 07:21:15.456865 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 07:21:15.456865 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 07:21:15.456865 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 07:21:15.456865 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 07:21:15.456865 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 07:21:15.456865 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 15 07:21:15.456865 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:21:15.456865 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 07:21:15.456865 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:21:15.456865 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:15.456865 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:15.456865 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:15.456865 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:15.456865 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:15.457922 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:15.457922 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:15.457922 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:15.457922 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:15.457922 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:15.457922 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:15.457922 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 07:21:15.457922 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 07:21:15.457922 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 07:21:15.457922 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 07:21:15.457922 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:15.457922 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:15.457922 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:15.457922 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:15.457922 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:15.457922 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:15.458744 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:15.458744 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:15.458744 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:15.458744 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:15.458744 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:15.458744 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:15.458744 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:15.458744 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:15.458744 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:15.458744 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:15.458744 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 07:21:15.458744 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 07:21:15.458744 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 15 07:21:15.458744 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:21:15.458744 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 417/850] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Sep 15 07:21:15 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:21:15.459564 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a933c62a-dbbe-4382-8d18-4353e7ca1ea4 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:15.551455 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd723f7-612e-4531-90e3-b78cde34657c None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:15.570332 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a933c62a-dbbe-4382-8d18-4353e7ca1ea4 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:15.589215 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dbbb9d-ba39-4462-8ab0-c91cac91f0e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:15.589958 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dbbb9d-ba39-4462-8ab0-c91cac91f0e8 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:15.590312 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dbbb9d-ba39-4462-8ab0-c91cac91f0e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:15.590656 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dbbb9d-ba39-4462-8ab0-c91cac91f0e8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:15.597835 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dbbb9d-ba39-4462-8ab0-c91cac91f0e8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:15.599285 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dbbb9d-ba39-4462-8ab0-c91cac91f0e8 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:15.606018 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d616dcc-d50f-491f-8339-e4e30616ec82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:15.606648 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d616dcc-d50f-491f-8339-e4e30616ec82 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:15.606934 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d616dcc-d50f-491f-8339-e4e30616ec82 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:15.607213 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d616dcc-d50f-491f-8339-e4e30616ec82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:15.611485 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-48dbbb9d-ba39-4462-8ab0-c91cac91f0e8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 15 07:21:15.611485 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:15.611485 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:15.611485 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:15.611485 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:15.611485 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:15.611485 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:15.611485 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:15.611485 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:15.611485 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:15.611485 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:15.611485 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:15.611485 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 07:21:15.611485 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 07:21:15.611485 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 07:21:15.611485 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 07:21:15.612409 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:15.612409 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:15.612409 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 07:21:15.612409 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 07:21:15.612409 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 07:21:15.612409 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 07:21:15.612409 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:15.612409 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:15.612409 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:15.612409 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:15.612409 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:15.612409 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:15.612409 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:15.612409 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:15.612409 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:15.612409 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:15.612409 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:15.612409 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:15.613342 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:15.613342 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:15.613342 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 07:21:15.613342 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 07:21:15.613342 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 07:21:15.613342 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 07:21:15.613342 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 07:21:15.613342 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 07:21:15.613342 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 15 07:21:15.613342 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:21:15.617353 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 434/851] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 15 07:21:15 2022] GET /identity/v3/users/swift => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:21:15.618230 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dbbb9d-ba39-4462-8ab0-c91cac91f0e8 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:15.619396 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d616dcc-d50f-491f-8339-e4e30616ec82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:15.620868 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d616dcc-d50f-491f-8339-e4e30616ec82 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:15.635936 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-3d616dcc-d50f-491f-8339-e4e30616ec82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:21:15.635936 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:15.635936 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:15.635936 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:15.635936 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:15.635936 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:15.635936 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:15.635936 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:15.635936 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:15.635936 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:15.635936 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:15.635936 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:15.635936 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 07:21:15.635936 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 07:21:15.635936 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 07:21:15.635936 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 07:21:15.636929 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:15.636929 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:15.636929 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:15.636929 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:15.636929 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:15.636929 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:15.636929 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:15.636929 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:15.636929 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:15.636929 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:15.636929 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:15.636929 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:15.636929 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:15.636929 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:15.636929 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:15.636929 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:15.636929 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 07:21:15.637857 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 07:21:15.637857 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:21:15.637857 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:21:15.637857 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 418/852] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:21:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:21:15.637857 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d616dcc-d50f-491f-8339-e4e30616ec82 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:15.654991 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:15.671191 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:15.735295 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dbbb9d-ba39-4462-8ab0-c91cac91f0e8 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:15.751301 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d616dcc-d50f-491f-8339-e4e30616ec82 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:15.775441 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b461f8-e515-4f6e-81c6-7ee108147775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:15.776521 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b461f8-e515-4f6e-81c6-7ee108147775 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:15.776809 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b461f8-e515-4f6e-81c6-7ee108147775 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:15.777344 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b461f8-e515-4f6e-81c6-7ee108147775 None admin] PATH_INFO: `/v3/services` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:15.778006 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b461f8-e515-4f6e-81c6-7ee108147775 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:15.778561 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b461f8-e515-4f6e-81c6-7ee108147775 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:15.780668 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b461f8-e515-4f6e-81c6-7ee108147775 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:15.786613 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94048cb-c30c-4b93-b403-8b30fd646a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:15.787216 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94048cb-c30c-4b93-b403-8b30fd646a86 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:15.787491 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94048cb-c30c-4b93-b403-8b30fd646a86 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:15.787764 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94048cb-c30c-4b93-b403-8b30fd646a86 None admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:15.788493 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94048cb-c30c-4b93-b403-8b30fd646a86 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:15.788795 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94048cb-c30c-4b93-b403-8b30fd646a86 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:15.790262 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94048cb-c30c-4b93-b403-8b30fd646a86 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:15.791098 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 435/853] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Sep 15 07:21:15 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:15.792235 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b461f8-e515-4f6e-81c6-7ee108147775 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:15.813563 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 419/854] 10.222.0.42 () {56 vars in 1135 bytes} [Thu Sep 15 07:21:15 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:15.813563 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94048cb-c30c-4b93-b403-8b30fd646a86 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:15.900227 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:15.909859 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:15.916020 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94048cb-c30c-4b93-b403-8b30fd646a86 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:15.918205 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b461f8-e515-4f6e-81c6-7ee108147775 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:15.956003 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740413ce-e7d8-426f-91d1-1394cfd12a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:15.956622 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740413ce-e7d8-426f-91d1-1394cfd12a67 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:15.956903 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740413ce-e7d8-426f-91d1-1394cfd12a67 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:15.957170 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740413ce-e7d8-426f-91d1-1394cfd12a67 None admin] PATH_INFO: `/v3/services` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:15.957292 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fea3262-46ed-4ba5-b334-739b746ee03e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:15.957821 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740413ce-e7d8-426f-91d1-1394cfd12a67 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:15.957922 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea3262-46ed-4ba5-b334-739b746ee03e None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:15.958124 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740413ce-e7d8-426f-91d1-1394cfd12a67 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:15.958219 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea3262-46ed-4ba5-b334-739b746ee03e None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:15.958397 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea3262-46ed-4ba5-b334-739b746ee03e None admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:15.959032 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fea3262-46ed-4ba5-b334-739b746ee03e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:15.959337 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fea3262-46ed-4ba5-b334-739b746ee03e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:15.959496 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740413ce-e7d8-426f-91d1-1394cfd12a67 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:15.960890 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fea3262-46ed-4ba5-b334-739b746ee03e None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:15.964491 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 420/855] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Sep 15 07:21:15 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:15.965229 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740413ce-e7d8-426f-91d1-1394cfd12a67 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:15.993895 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 436/856] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 15 07:21:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:15.994897 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fea3262-46ed-4ba5-b334-739b746ee03e None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:16.108312 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740413ce-e7d8-426f-91d1-1394cfd12a67 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:16.119170 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fea3262-46ed-4ba5-b334-739b746ee03e None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:16.148627 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98f15da-6872-4051-b9ab-1f6f45ede2ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:16.148858 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98f15da-6872-4051-b9ab-1f6f45ede2ac None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:16.149129 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98f15da-6872-4051-b9ab-1f6f45ede2ac None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:16.149407 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98f15da-6872-4051-b9ab-1f6f45ede2ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:16.165846 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4dd7d8-9088-44e5-83bc-3301acf9bffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:16.167119 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4dd7d8-9088-44e5-83bc-3301acf9bffe None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:16.167475 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4dd7d8-9088-44e5-83bc-3301acf9bffe None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:16.167944 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4dd7d8-9088-44e5-83bc-3301acf9bffe None admin] PATH_INFO: `/v3/projects/service` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:16.169872 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:16.176348 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4dd7d8-9088-44e5-83bc-3301acf9bffe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:16.177859 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4dd7d8-9088-44e5-83bc-3301acf9bffe None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:16.191817 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-0b4dd7d8-9088-44e5-83bc-3301acf9bffe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 07:21:16.191817 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:16.191817 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:16.191817 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:16.191817 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:16.191817 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:16.191817 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:16.191817 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:16.191817 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:16.191817 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:16.191817 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:16.191817 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:16.191817 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 07:21:16.191817 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 07:21:16.191817 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 07:21:16.192765 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 07:21:16.192765 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:16.192765 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:16.192765 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:16.192765 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:16.192765 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:16.192765 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:16.192765 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:16.192765 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:16.192765 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:16.192765 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:16.192765 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:16.192765 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:16.192765 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:16.192765 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:16.192765 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:16.192765 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:16.193729 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 07:21:16.193729 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 07:21:16.193729 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 07:21:16.193729 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 07:21:16.193729 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 07:21:16.193729 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 07:21:16.193729 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 07:21:16.193729 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:21:16.197882 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 437/857] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 15 07:21:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 07:21:16.198817 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4dd7d8-9088-44e5-83bc-3301acf9bffe None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:16.216004 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:16.224301 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98f15da-6872-4051-b9ab-1f6f45ede2ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:16.225864 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98f15da-6872-4051-b9ab-1f6f45ede2ac None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:16.312554 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4dd7d8-9088-44e5-83bc-3301acf9bffe None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:16.353131 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42abc948-90d9-459b-8f84-b3d28ecf6d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:16.353805 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42abc948-90d9-459b-8f84-b3d28ecf6d26 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:16.354086 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42abc948-90d9-459b-8f84-b3d28ecf6d26 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:16.354363 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42abc948-90d9-459b-8f84-b3d28ecf6d26 None admin] PATH_INFO: `/v3/projects` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:16.355070 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42abc948-90d9-459b-8f84-b3d28ecf6d26 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:16.355375 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42abc948-90d9-459b-8f84-b3d28ecf6d26 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:16.356893 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42abc948-90d9-459b-8f84-b3d28ecf6d26 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:16.366280 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 421/858] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 15 07:21:15 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 07:21:16.370597 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 438/859] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 15 07:21:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:16.372124 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98f15da-6872-4051-b9ab-1f6f45ede2ac None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:16.376278 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42abc948-90d9-459b-8f84-b3d28ecf6d26 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:16.438306 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:16.452837 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:16.488652 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42abc948-90d9-459b-8f84-b3d28ecf6d26 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:16.505597 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98f15da-6872-4051-b9ab-1f6f45ede2ac None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:16.525408 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99865a2b-a78a-43b0-a174-92ab7c08eee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:16.526055 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99865a2b-a78a-43b0-a174-92ab7c08eee1 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:16.526336 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99865a2b-a78a-43b0-a174-92ab7c08eee1 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:16.526611 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99865a2b-a78a-43b0-a174-92ab7c08eee1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:16.527340 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99865a2b-a78a-43b0-a174-92ab7c08eee1 None admin] RBAC: Adding query filter params (role.id=0e661b5a67664692b933e5da9fc2e488, scope.project.id=4e9987720930473da840579654311bc2, user.id=f9c68c4bd9074d8d9b893755e28c0807) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:16.527654 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99865a2b-a78a-43b0-a174-92ab7c08eee1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:16.529089 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99865a2b-a78a-43b0-a174-92ab7c08eee1 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:16.545610 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 439/860] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 15 07:21:16 2022] GET /identity/v3/role_assignments?user.id=f9c68c4bd9074d8d9b893755e28c0807&scope.project.id=4e9987720930473da840579654311bc2&role.id=0e661b5a67664692b933e5da9fc2e488 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:21:16.545610 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcb87fb-9258-4eb3-b6de-b8951a10fa34 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:16.545610 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcb87fb-9258-4eb3-b6de-b8951a10fa34 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:16.545610 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcb87fb-9258-4eb3-b6de-b8951a10fa34 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:16.545610 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 440/861] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:21:16 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:21:16.545610 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bc6abf-36b3-4313-a93c-b2170600c800 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:16.546306 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bc6abf-36b3-4313-a93c-b2170600c800 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:16.546306 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bc6abf-36b3-4313-a93c-b2170600c800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:16.556124 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbadb052-bdcc-4688-9719-37b0d856bfd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:16.556971 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbadb052-bdcc-4688-9719-37b0d856bfd1 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:16.557345 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbadb052-bdcc-4688-9719-37b0d856bfd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:16.557758 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbadb052-bdcc-4688-9719-37b0d856bfd1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:16.565333 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-23bc6abf-36b3-4313-a93c-b2170600c800 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:21:16.566722 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-23bc6abf-36b3-4313-a93c-b2170600c800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:21:16.570594 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbadb052-bdcc-4688-9719-37b0d856bfd1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:16.572179 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbadb052-bdcc-4688-9719-37b0d856bfd1 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:16.590617 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-fbadb052-bdcc-4688-9719-37b0d856bfd1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:21:16.590617 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:16.590617 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:16.590617 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:16.590617 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:16.590617 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:16.590617 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:16.590617 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:16.590617 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:16.590617 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:16.590617 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:16.590617 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:16.590617 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 07:21:16.590617 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 07:21:16.590617 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 07:21:16.590617 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 07:21:16.591652 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:16.591652 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:16.591652 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:16.591652 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:16.591652 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:16.591652 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:16.591652 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:16.591652 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:16.591652 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:16.591652 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:16.591652 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:16.591652 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:16.591652 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:16.591652 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:16.591652 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:16.591652 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:16.591652 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 07:21:16.592541 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 07:21:16.592541 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:21:16.592541 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:21:16.596873 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 422/862] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:21:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 07:21:16.598058 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbadb052-bdcc-4688-9719-37b0d856bfd1 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:16.658034 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 441/863] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:21:16 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 07:21:16.717385 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbadb052-bdcc-4688-9719-37b0d856bfd1 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:16.719765 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:16.760513 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faf40b3-d03a-4987-a76b-e5da73ce72c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:16.761127 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faf40b3-d03a-4987-a76b-e5da73ce72c6 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:16.761410 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faf40b3-d03a-4987-a76b-e5da73ce72c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:16.761717 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faf40b3-d03a-4987-a76b-e5da73ce72c6 None admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:16.762363 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faf40b3-d03a-4987-a76b-e5da73ce72c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:16.762672 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faf40b3-d03a-4987-a76b-e5da73ce72c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:16.764055 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faf40b3-d03a-4987-a76b-e5da73ce72c6 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:16.777212 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 423/864] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 15 07:21:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:16.783320 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bc6abf-36b3-4313-a93c-b2170600c800 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:16.861154 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:16.896966 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bc6abf-36b3-4313-a93c-b2170600c800 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:16.937730 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff1f794-0da3-47b8-918f-63a104cd80fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:16.939170 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff1f794-0da3-47b8-918f-63a104cd80fc None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:16.939657 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff1f794-0da3-47b8-918f-63a104cd80fc None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:16.940158 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff1f794-0da3-47b8-918f-63a104cd80fc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:16.950275 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff1f794-0da3-47b8-918f-63a104cd80fc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:16.950275 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff1f794-0da3-47b8-918f-63a104cd80fc None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:16.959771 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faf40b3-d03a-4987-a76b-e5da73ce72c6 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:16.961977 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-5ff1f794-0da3-47b8-918f-63a104cd80fc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 07:21:16.961977 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:16.961977 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:16.961977 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:16.961977 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:16.961977 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:16.961977 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:16.961977 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:16.961977 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:16.961977 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:16.961977 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:16.961977 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:16.961977 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 07:21:16.961977 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 07:21:16.961977 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 07:21:16.962840 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 07:21:16.962840 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:16.962840 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:16.962840 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:16.962840 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:16.962840 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:16.962840 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:16.962840 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:16.962840 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:16.962840 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:16.962840 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:16.962840 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:16.962840 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:16.962840 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:16.962840 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:16.962840 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:16.962840 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:16.963716 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 07:21:16.963716 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 07:21:16.963716 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 07:21:16.963716 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 07:21:16.963716 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 07:21:16.963716 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 07:21:16.963716 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 07:21:16.963716 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:21:16.963716 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 442/865] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 15 07:21:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 07:21:16.969015 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff1f794-0da3-47b8-918f-63a104cd80fc None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:17.079602 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faf40b3-d03a-4987-a76b-e5da73ce72c6 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:17.081526 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff1f794-0da3-47b8-918f-63a104cd80fc None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:17.109242 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:17.119354 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25118cf-9f00-48d2-a0aa-a7d35daee9ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:17.119567 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db146ca0-820a-41aa-be59-61b5bd75f072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:17.119995 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25118cf-9f00-48d2-a0aa-a7d35daee9ec None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:17.120278 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25118cf-9f00-48d2-a0aa-a7d35daee9ec None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:17.120551 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25118cf-9f00-48d2-a0aa-a7d35daee9ec None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:17.120798 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db146ca0-820a-41aa-be59-61b5bd75f072 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:17.121133 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25118cf-9f00-48d2-a0aa-a7d35daee9ec None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:17.121373 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db146ca0-820a-41aa-be59-61b5bd75f072 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:17.121816 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db146ca0-820a-41aa-be59-61b5bd75f072 None admin] PATH_INFO: `/v3/projects` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:17.122513 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25118cf-9f00-48d2-a0aa-a7d35daee9ec None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:17.122828 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db146ca0-820a-41aa-be59-61b5bd75f072 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:17.123280 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db146ca0-820a-41aa-be59-61b5bd75f072 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:17.125040 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db146ca0-820a-41aa-be59-61b5bd75f072 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:17.133840 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-d25118cf-9f00-48d2-a0aa-a7d35daee9ec None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 15 07:21:17.133840 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:17.133840 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 07:21:17.133840 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 07:21:17.133840 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 07:21:17.133840 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 07:21:17.133840 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 07:21:17.133840 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 07:21:17.133840 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 15 07:21:17.133840 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:21:17.133840 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 07:21:17.133840 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:21:17.133840 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:17.133840 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:17.133840 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:17.133840 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:17.135097 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:17.135097 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:17.135097 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:17.135097 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:17.135097 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:17.135097 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:17.135097 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:17.135097 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 07:21:17.135097 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 07:21:17.135097 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 07:21:17.135097 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 07:21:17.135097 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:17.135097 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:17.135097 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:17.135097 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:17.135097 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:17.135097 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:17.135995 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:17.135995 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:17.135995 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:17.135995 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:17.135995 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:17.135995 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:17.135995 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:17.135995 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:17.135995 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:17.135995 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:17.135995 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 07:21:17.135995 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 07:21:17.135995 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 15 07:21:17.135995 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:21:17.135995 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 424/866] 10.222.0.42 () {56 vars in 1132 bytes} [Thu Sep 15 07:21:16 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 07:21:17.140615 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25118cf-9f00-48d2-a0aa-a7d35daee9ec None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:17.153164 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 443/867] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 15 07:21:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:21:17.154234 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db146ca0-820a-41aa-be59-61b5bd75f072 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:17.181107 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:17.277177 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25118cf-9f00-48d2-a0aa-a7d35daee9ec None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:17.280504 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db146ca0-820a-41aa-be59-61b5bd75f072 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:17.317751 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d814679-6983-4004-b92c-f9cbaaad03e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:17.318455 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d814679-6983-4004-b92c-f9cbaaad03e2 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:17.318803 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d814679-6983-4004-b92c-f9cbaaad03e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:17.319161 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d814679-6983-4004-b92c-f9cbaaad03e2 None admin] PATH_INFO: `/v3/services` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:17.319857 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d814679-6983-4004-b92c-f9cbaaad03e2 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:17.320221 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d814679-6983-4004-b92c-f9cbaaad03e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:17.321028 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dc176f-9f65-4037-87ce-73639416615f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:17.321888 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dc176f-9f65-4037-87ce-73639416615f None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:17.322344 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dc176f-9f65-4037-87ce-73639416615f None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:17.323671 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d814679-6983-4004-b92c-f9cbaaad03e2 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:17.323978 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dc176f-9f65-4037-87ce-73639416615f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:17.325353 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dc176f-9f65-4037-87ce-73639416615f None admin] RBAC: Adding query filter params (role.id=6bbb643e0d8d4bc0a9b27c26edf59216, scope.project.id=4e9987720930473da840579654311bc2, user.id=72b90d35087e4d7a80e9417082a1541f) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:17.325902 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dc176f-9f65-4037-87ce-73639416615f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:17.327810 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dc176f-9f65-4037-87ce-73639416615f None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:17.333103 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 425/868] 10.222.0.42 () {56 vars in 1126 bytes} [Thu Sep 15 07:21:17 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:17.334562 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d814679-6983-4004-b92c-f9cbaaad03e2 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:17.346772 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 444/869] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 15 07:21:17 2022] GET /identity/v3/role_assignments?user.id=72b90d35087e4d7a80e9417082a1541f&scope.project.id=4e9987720930473da840579654311bc2&role.id=6bbb643e0d8d4bc0a9b27c26edf59216 => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:17.449533 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:17.459230 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d814679-6983-4004-b92c-f9cbaaad03e2 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:17.497878 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fe25d8-9ed2-4220-a78b-3ce6828bbc55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:17.498910 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fe25d8-9ed2-4220-a78b-3ce6828bbc55 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:17.499411 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fe25d8-9ed2-4220-a78b-3ce6828bbc55 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:17.499925 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fe25d8-9ed2-4220-a78b-3ce6828bbc55 None admin] PATH_INFO: `/v3/services` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:17.501365 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fe25d8-9ed2-4220-a78b-3ce6828bbc55 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:17.501920 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fe25d8-9ed2-4220-a78b-3ce6828bbc55 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:17.503550 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fe25d8-9ed2-4220-a78b-3ce6828bbc55 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:17.509771 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 426/870] 10.222.0.42 () {56 vars in 1126 bytes} [Thu Sep 15 07:21:17 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:18.187391 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6a6502-b260-4470-a050-685009a2df51 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:18.188589 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6a6502-b260-4470-a050-685009a2df51 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:18.188806 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6a6502-b260-4470-a050-685009a2df51 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:18.190400 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 445/871] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:21:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:21:18.198144 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f451ec-1deb-4802-b6c2-70854815d7f1 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:18.198553 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f451ec-1deb-4802-b6c2-70854815d7f1 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:18.198814 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f451ec-1deb-4802-b6c2-70854815d7f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:18.218578 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-91f451ec-1deb-4802-b6c2-70854815d7f1 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:21:18.220419 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-91f451ec-1deb-4802-b6c2-70854815d7f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:21:18.293100 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 427/872] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:21:18 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 07:21:18.564500 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6a6502-b260-4470-a050-685009a2df51 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:18.617527 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:18.671432 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6a6502-b260-4470-a050-685009a2df51 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:18.712289 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fb9037-7c97-446f-959e-481ac2c9ba95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:18.712915 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb9037-7c97-446f-959e-481ac2c9ba95 None admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:18.713192 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb9037-7c97-446f-959e-481ac2c9ba95 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:18.713470 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb9037-7c97-446f-959e-481ac2c9ba95 None admin] PATH_INFO: `/v3/services` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:18.714164 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fb9037-7c97-446f-959e-481ac2c9ba95 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:18.715531 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fb9037-7c97-446f-959e-481ac2c9ba95 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:18.798369 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 446/873] 10.222.0.42 () {60 vars in 1141 bytes} [Thu Sep 15 07:21:18 2022] POST /identity/v3/services => generated 251 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:21:18.944850 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892269f1-cc15-4ace-9afb-d4c693647e77 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:18.945144 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892269f1-cc15-4ace-9afb-d4c693647e77 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:18.945421 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892269f1-cc15-4ace-9afb-d4c693647e77 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:18.947089 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 428/874] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:21:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:21:18.952560 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7398b435-f12d-4fcd-bd95-7a2f19f65cd7 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:18.952854 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7398b435-f12d-4fcd-bd95-7a2f19f65cd7 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:18.953134 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7398b435-f12d-4fcd-bd95-7a2f19f65cd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:18.973313 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-7398b435-f12d-4fcd-bd95-7a2f19f65cd7 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:21:18.974643 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-7398b435-f12d-4fcd-bd95-7a2f19f65cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:21:19.055949 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 447/875] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:21:18 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 07:21:19.334571 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892269f1-cc15-4ace-9afb-d4c693647e77 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:19.423308 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:19.443009 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892269f1-cc15-4ace-9afb-d4c693647e77 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:19.478974 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e61bc1-08df-49f0-accd-7970e18bb706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:19.479608 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e61bc1-08df-49f0-accd-7970e18bb706 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:19.479890 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e61bc1-08df-49f0-accd-7970e18bb706 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:19.480169 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e61bc1-08df-49f0-accd-7970e18bb706 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:19.491081 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e61bc1-08df-49f0-accd-7970e18bb706 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:21:19.495915 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-f4e61bc1-08df-49f0-accd-7970e18bb706 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 15 07:21:19.495915 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:19.495915 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:19.495915 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:19.495915 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:19.495915 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:19.495915 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:19.495915 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:19.495915 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:19.495915 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:19.495915 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:19.495915 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:19.495915 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:21:19.495915 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:21:19.495915 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:21:19.495915 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:21:19.498265 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:21:19.498265 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:21:19.498265 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:21:19.498265 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:21:19.498265 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:19.498265 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:19.498265 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:19.498265 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:19.498265 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:19.498265 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:19.498265 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:19.498265 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:19.498265 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:19.498265 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:19.498265 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:19.498265 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:19.500264 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:19.500264 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:19.500264 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:19.500264 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:19.500264 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:21:19.500264 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:21:19.500264 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:21:19.500264 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:21:19.500264 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:21:19.500264 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:21:19.500264 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 15 07:21:19.500264 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:21:19.500264 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 429/876] 10.222.0.42 () {56 vars in 1120 bytes} [Thu Sep 15 07:21:19 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:21:19.505625 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7398b435-f12d-4fcd-bd95-7a2f19f65cd7 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:19.543096 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:19.623572 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7398b435-f12d-4fcd-bd95-7a2f19f65cd7 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:19.670890 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e352937-7140-4871-99a9-e8a29ac1228d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:19.671497 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e352937-7140-4871-99a9-e8a29ac1228d None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:19.673607 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e352937-7140-4871-99a9-e8a29ac1228d None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:19.673607 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e352937-7140-4871-99a9-e8a29ac1228d None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:19.673607 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e352937-7140-4871-99a9-e8a29ac1228d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:19.673607 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e352937-7140-4871-99a9-e8a29ac1228d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:19.674448 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e352937-7140-4871-99a9-e8a29ac1228d None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:19.683240 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 448/877] 10.222.0.42 () {56 vars in 1115 bytes} [Thu Sep 15 07:21:19 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:19.687362 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cd7e24-b655-4132-bfcd-68a39588f9a8 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:19.687655 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cd7e24-b655-4132-bfcd-68a39588f9a8 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:19.688481 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cd7e24-b655-4132-bfcd-68a39588f9a8 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:19.690245 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e352937-7140-4871-99a9-e8a29ac1228d None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:19.690702 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 430/878] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:21:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 07:21:19.696212 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fa7ef4-4ea1-4887-b3ad-2c955c7cc8c6 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:19.696496 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fa7ef4-4ea1-4887-b3ad-2c955c7cc8c6 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:19.696775 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fa7ef4-4ea1-4887-b3ad-2c955c7cc8c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:19.715260 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-51fa7ef4-4ea1-4887-b3ad-2c955c7cc8c6 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:21:19.716464 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-51fa7ef4-4ea1-4887-b3ad-2c955c7cc8c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:21:19.788913 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 431/879] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:21:19 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 07:21:19.796925 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:19.799489 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7717ed1-d76f-45ed-81af-b4aab5145b19 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:19.799642 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7717ed1-d76f-45ed-81af-b4aab5145b19 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:19.799932 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7717ed1-d76f-45ed-81af-b4aab5145b19 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:19.805681 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 432/880] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:21:19 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:21:19.812902 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e352937-7140-4871-99a9-e8a29ac1228d None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:19.813173 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6831f6-e38d-4ff9-aa30-07a357123f3f None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:19.813173 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6831f6-e38d-4ff9-aa30-07a357123f3f None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:19.813173 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6831f6-e38d-4ff9-aa30-07a357123f3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:19.828238 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-af6831f6-e38d-4ff9-aa30-07a357123f3f None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:21:19.830350 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-af6831f6-e38d-4ff9-aa30-07a357123f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:21:19.851444 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5b8498-a56e-445b-a733-3f3ad6ff3960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:19.852083 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5b8498-a56e-445b-a733-3f3ad6ff3960 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:19.852366 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5b8498-a56e-445b-a733-3f3ad6ff3960 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:19.852649 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5b8498-a56e-445b-a733-3f3ad6ff3960 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:19.864142 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5b8498-a56e-445b-a733-3f3ad6ff3960 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:19.865681 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5b8498-a56e-445b-a733-3f3ad6ff3960 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:19.881237 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-7d5b8498-a56e-445b-a733-3f3ad6ff3960 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:21:19.881237 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:19.881237 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:19.881237 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:19.881237 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:19.881237 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:19.881237 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:19.881237 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:19.881237 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:19.881237 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:19.881237 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:19.881237 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:19.881237 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 07:21:19.881237 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 07:21:19.881237 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 07:21:19.881237 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 07:21:19.882370 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:19.882370 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:19.882370 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:19.882370 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:19.882370 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:19.882370 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:19.882370 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:19.882370 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:19.882370 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:19.882370 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:19.882370 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:19.882370 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:19.882370 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:19.882370 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:19.882370 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:19.882370 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:19.882370 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 07:21:19.883380 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 07:21:19.883380 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:21:19.883380 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:21:19.883380 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 449/881] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:21:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 07:21:19.890089 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5b8498-a56e-445b-a733-3f3ad6ff3960 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:19.925759 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 433/882] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:21:19 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 07:21:20.012424 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5b8498-a56e-445b-a733-3f3ad6ff3960 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:20.047830 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ff99f6-c333-4415-902f-111e12732322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:20.048487 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ff99f6-c333-4415-902f-111e12732322 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:20.048767 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ff99f6-c333-4415-902f-111e12732322 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:20.049090 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ff99f6-c333-4415-902f-111e12732322 None admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:20.049765 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ff99f6-c333-4415-902f-111e12732322 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:20.050126 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ff99f6-c333-4415-902f-111e12732322 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:20.051494 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ff99f6-c333-4415-902f-111e12732322 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:20.064588 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 450/883] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 15 07:21:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:20.070711 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6831f6-e38d-4ff9-aa30-07a357123f3f None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:20.073664 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:20.081938 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ff99f6-c333-4415-902f-111e12732322 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:20.094152 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:20.195956 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6831f6-e38d-4ff9-aa30-07a357123f3f None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:20.213355 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ff99f6-c333-4415-902f-111e12732322 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:20.240753 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a5d0d3-0c04-45d6-be91-c71f7bea81db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:20.241398 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a5d0d3-0c04-45d6-be91-c71f7bea81db None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:20.241711 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a5d0d3-0c04-45d6-be91-c71f7bea81db None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:20.241986 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a5d0d3-0c04-45d6-be91-c71f7bea81db None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:20.249437 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a5d0d3-0c04-45d6-be91-c71f7bea81db None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:20.250614 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3137fb1-1656-4862-9132-4f7552d745cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:20.250956 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a5d0d3-0c04-45d6-be91-c71f7bea81db None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:20.251441 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3137fb1-1656-4862-9132-4f7552d745cc None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:20.251742 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3137fb1-1656-4862-9132-4f7552d745cc None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:20.252019 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3137fb1-1656-4862-9132-4f7552d745cc None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:20.252595 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3137fb1-1656-4862-9132-4f7552d745cc None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:20.254018 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3137fb1-1656-4862-9132-4f7552d745cc None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:20.262702 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-b3137fb1-1656-4862-9132-4f7552d745cc None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 07:21:20.262702 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:20.262702 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 07:21:20.262702 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 07:21:20.262702 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 07:21:20.262702 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 07:21:20.262702 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 07:21:20.262702 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 07:21:20.262702 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 07:21:20.262702 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:21:20.262702 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 07:21:20.262702 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:21:20.262702 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:20.262702 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:20.262702 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:20.262702 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:20.262702 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:20.263719 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:20.263719 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:20.263719 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:20.263719 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:20.263719 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:20.263719 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:20.263719 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 07:21:20.263719 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 07:21:20.263719 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 07:21:20.263719 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 07:21:20.263719 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:20.263719 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:20.263719 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:20.263719 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:20.263719 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:20.263719 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:20.265177 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:20.265177 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:20.265177 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:20.265177 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:20.265177 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:20.265177 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:20.265177 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:20.265177 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:20.265177 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:20.265177 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:20.265177 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 07:21:20.265177 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 07:21:20.265177 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 07:21:20.265177 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:21:20.266160 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-92a5d0d3-0c04-45d6-be91-c71f7bea81db None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 15 07:21:20.266160 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:20.266160 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:20.266160 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:20.266160 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:20.266160 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:20.266160 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:20.266160 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:20.266160 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:20.266160 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:20.266160 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:20.266160 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:20.266160 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 07:21:20.266160 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 07:21:20.266160 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 07:21:20.266160 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 07:21:20.266994 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:20.266994 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:20.266994 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 07:21:20.266994 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 07:21:20.266994 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 07:21:20.266994 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 07:21:20.266994 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:20.266994 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:20.266994 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:20.266994 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:20.266994 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:20.266994 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:20.266994 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:20.266994 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:20.266994 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:20.266994 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:20.266994 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:20.266994 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:20.267927 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:20.267927 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:20.267927 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 07:21:20.267927 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 07:21:20.267927 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 07:21:20.267927 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 07:21:20.267927 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 07:21:20.267927 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 07:21:20.267927 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 15 07:21:20.267927 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:21:20.269228 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 451/884] 10.222.0.42 () {56 vars in 1126 bytes} [Thu Sep 15 07:21:20 2022] GET /identity/v3/services/object-store => generated 93 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 07:21:20.269228 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3137fb1-1656-4862-9132-4f7552d745cc None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:20.269622 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 434/885] 10.222.0.42 () {56 vars in 1117 bytes} [Thu Sep 15 07:21:20 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:21:20.273759 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a5d0d3-0c04-45d6-be91-c71f7bea81db None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:20.336334 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:20.349769 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:20.377246 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3137fb1-1656-4862-9132-4f7552d745cc None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:20.380052 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a5d0d3-0c04-45d6-be91-c71f7bea81db None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:20.411937 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f61a85-4218-44ad-8963-7c971b610286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:20.412650 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f61a85-4218-44ad-8963-7c971b610286 None admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:20.412938 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f61a85-4218-44ad-8963-7c971b610286 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:20.413212 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f61a85-4218-44ad-8963-7c971b610286 None admin] PATH_INFO: `/v3/services` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:20.413950 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f61a85-4218-44ad-8963-7c971b610286 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:20.415310 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f61a85-4218-44ad-8963-7c971b610286 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:20.418336 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceb8164-623b-496c-933d-9970d9928a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:20.419117 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceb8164-623b-496c-933d-9970d9928a20 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:20.419569 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceb8164-623b-496c-933d-9970d9928a20 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:20.419846 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceb8164-623b-496c-933d-9970d9928a20 None admin] PATH_INFO: `/v3/services` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:20.420478 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eceb8164-623b-496c-933d-9970d9928a20 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:20.420787 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eceb8164-623b-496c-933d-9970d9928a20 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:20.422154 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eceb8164-623b-496c-933d-9970d9928a20 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:20.428256 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 435/886] 10.222.0.42 () {56 vars in 1122 bytes} [Thu Sep 15 07:21:20 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:20.429010 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceb8164-623b-496c-933d-9970d9928a20 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:20.517509 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 452/887] 10.222.0.42 () {60 vars in 1141 bytes} [Thu Sep 15 07:21:20 2022] POST /identity/v3/services => generated 269 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:21:20.519023 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f61a85-4218-44ad-8963-7c971b610286 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:20.543320 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceb8164-623b-496c-933d-9970d9928a20 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:20.581883 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:20.601734 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91210f48-dece-44f6-ac90-709c9026a22e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:20.602364 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91210f48-dece-44f6-ac90-709c9026a22e None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:20.602647 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91210f48-dece-44f6-ac90-709c9026a22e None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:20.602923 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91210f48-dece-44f6-ac90-709c9026a22e None admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:20.603673 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91210f48-dece-44f6-ac90-709c9026a22e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:20.603983 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91210f48-dece-44f6-ac90-709c9026a22e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:20.605741 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91210f48-dece-44f6-ac90-709c9026a22e None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:20.633081 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 436/888] 10.222.0.42 () {56 vars in 1149 bytes} [Thu Sep 15 07:21:20 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:20.635844 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91210f48-dece-44f6-ac90-709c9026a22e None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:20.678361 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f61a85-4218-44ad-8963-7c971b610286 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:20.719884 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a1125e-3f49-46e4-a9fa-c5ac00225f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:20.720501 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a1125e-3f49-46e4-a9fa-c5ac00225f72 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:20.720803 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a1125e-3f49-46e4-a9fa-c5ac00225f72 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:20.721045 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a1125e-3f49-46e4-a9fa-c5ac00225f72 None admin] PATH_INFO: `/v3/services` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:20.721721 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a1125e-3f49-46e4-a9fa-c5ac00225f72 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:20.722018 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a1125e-3f49-46e4-a9fa-c5ac00225f72 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:20.723372 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a1125e-3f49-46e4-a9fa-c5ac00225f72 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:20.724682 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:20.733651 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 453/889] 10.222.0.42 () {56 vars in 1122 bytes} [Thu Sep 15 07:21:20 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:20.777696 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91210f48-dece-44f6-ac90-709c9026a22e None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:20.818791 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7d5fac-faaf-492b-a915-9955d1bb1e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:20.819690 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7d5fac-faaf-492b-a915-9955d1bb1e3d None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:20.820138 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7d5fac-faaf-492b-a915-9955d1bb1e3d None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:20.820564 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7d5fac-faaf-492b-a915-9955d1bb1e3d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:20.876542 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:20.898681 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7d5fac-faaf-492b-a915-9955d1bb1e3d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:20.900358 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7d5fac-faaf-492b-a915-9955d1bb1e3d None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:21.018602 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 437/890] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 15 07:21:20 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 15 07:21:21.025608 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a1125e-3f49-46e4-a9fa-c5ac00225f72 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:21.139569 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd5a1f9-7719-457a-a2c3-afe6ceb09181 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:21.139946 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd5a1f9-7719-457a-a2c3-afe6ceb09181 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:21.140058 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a1125e-3f49-46e4-a9fa-c5ac00225f72 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:21.140155 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd5a1f9-7719-457a-a2c3-afe6ceb09181 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:21.141907 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 438/891] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:21:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 07:21:21.147081 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21098aec-4715-4c7c-829e-24ed33f6d079 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:21.147371 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21098aec-4715-4c7c-829e-24ed33f6d079 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:21.147653 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21098aec-4715-4c7c-829e-24ed33f6d079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:21.167088 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-21098aec-4715-4c7c-829e-24ed33f6d079 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:21:21.168340 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-21098aec-4715-4c7c-829e-24ed33f6d079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:21:21.182042 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bfac12-0397-432d-90d6-461901131afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:21.182688 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bfac12-0397-432d-90d6-461901131afe None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:21.183218 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bfac12-0397-432d-90d6-461901131afe None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:21.183491 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bfac12-0397-432d-90d6-461901131afe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:21.200048 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bfac12-0397-432d-90d6-461901131afe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:21.202569 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bfac12-0397-432d-90d6-461901131afe None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:21.221409 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-a5bfac12-0397-432d-90d6-461901131afe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:21:21.221409 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:21.221409 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:21.221409 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:21.221409 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:21.221409 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:21.221409 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:21.221409 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:21.221409 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:21.221409 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:21.221409 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:21.221409 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:21.221409 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 07:21:21.221409 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 07:21:21.221409 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 07:21:21.221409 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 07:21:21.222428 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:21.222428 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:21.222428 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:21.222428 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:21.222428 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:21.222428 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:21.222428 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:21.222428 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:21.222428 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:21.222428 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:21.222428 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:21.222428 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:21.222428 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:21.222428 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:21.222428 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:21.222428 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:21.222428 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 07:21:21.223305 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 07:21:21.223305 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:21:21.223305 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:21:21.223305 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 454/892] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:21:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:21:21.228168 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bfac12-0397-432d-90d6-461901131afe None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:21.257707 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 439/893] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:21:21 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 07:21:21.287055 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:21.340126 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bfac12-0397-432d-90d6-461901131afe None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:21.376499 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f38773e-cf3b-4930-ad8b-5e7129faaf29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:21.377362 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f38773e-cf3b-4930-ad8b-5e7129faaf29 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:21.377843 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f38773e-cf3b-4930-ad8b-5e7129faaf29 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:21.378262 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f38773e-cf3b-4930-ad8b-5e7129faaf29 None admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:21.379069 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f38773e-cf3b-4930-ad8b-5e7129faaf29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:21.379527 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f38773e-cf3b-4930-ad8b-5e7129faaf29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:21.381051 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f38773e-cf3b-4930-ad8b-5e7129faaf29 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:21.394842 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 455/894] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 15 07:21:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:21.405626 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21098aec-4715-4c7c-829e-24ed33f6d079 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:21.413526 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:21.509534 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21098aec-4715-4c7c-829e-24ed33f6d079 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:21.545633 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fbd983-7fde-4e7f-9a4c-55673eb69a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:21.546301 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fbd983-7fde-4e7f-9a4c-55673eb69a70 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:21.546586 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fbd983-7fde-4e7f-9a4c-55673eb69a70 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:21.546864 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fbd983-7fde-4e7f-9a4c-55673eb69a70 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:21.555925 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fbd983-7fde-4e7f-9a4c-55673eb69a70 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:21.556144 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f38773e-cf3b-4930-ad8b-5e7129faaf29 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:21.557394 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fbd983-7fde-4e7f-9a4c-55673eb69a70 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:21.570249 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-b7fbd983-7fde-4e7f-9a4c-55673eb69a70 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 07:21:21.570249 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:21.570249 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:21.570249 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:21.570249 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:21.570249 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:21.570249 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:21.570249 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:21.570249 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:21.570249 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:21.570249 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:21.570249 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:21.570249 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 07:21:21.570249 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 07:21:21.570249 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 07:21:21.571214 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 07:21:21.571214 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:21.571214 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:21.571214 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:21.571214 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:21.571214 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:21.571214 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:21.571214 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:21.571214 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:21.571214 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:21.571214 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:21.571214 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:21.571214 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:21.571214 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:21.571214 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:21.571214 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:21.571214 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:21.572090 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 07:21:21.572090 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 07:21:21.572090 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 07:21:21.572090 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 07:21:21.572090 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 07:21:21.572090 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 07:21:21.572090 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 07:21:21.572090 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:21:21.572090 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 440/895] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 15 07:21:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:21:21.577004 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fbd983-7fde-4e7f-9a4c-55673eb69a70 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:21.649144 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:21.670933 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f38773e-cf3b-4930-ad8b-5e7129faaf29 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:21.680718 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fbd983-7fde-4e7f-9a4c-55673eb69a70 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:21.683309 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:21.709707 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acce374f-7070-4891-888a-75745ac5e0e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:21.710972 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acce374f-7070-4891-888a-75745ac5e0e8 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:21.712698 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acce374f-7070-4891-888a-75745ac5e0e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:21.712698 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acce374f-7070-4891-888a-75745ac5e0e8 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:21.712698 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acce374f-7070-4891-888a-75745ac5e0e8 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:21.714463 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acce374f-7070-4891-888a-75745ac5e0e8 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:21.716127 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc206b8-040f-48a8-a321-08ff792dec20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:21.716830 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc206b8-040f-48a8-a321-08ff792dec20 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:21.717189 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc206b8-040f-48a8-a321-08ff792dec20 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:21.717540 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc206b8-040f-48a8-a321-08ff792dec20 None admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:21.718348 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc206b8-040f-48a8-a321-08ff792dec20 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:21.718733 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc206b8-040f-48a8-a321-08ff792dec20 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:21.720966 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc206b8-040f-48a8-a321-08ff792dec20 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:21.725800 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-acce374f-7070-4891-888a-75745ac5e0e8 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 15 07:21:21.725800 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:21.725800 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 07:21:21.725800 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 07:21:21.725800 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 07:21:21.725800 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 07:21:21.725800 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 07:21:21.725800 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 07:21:21.725800 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 15 07:21:21.725800 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:21:21.725800 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 07:21:21.725800 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:21:21.725800 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:21.725800 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:21.725800 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:21.725800 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:21.725800 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:21.726979 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:21.726979 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:21.726979 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:21.726979 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:21.726979 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:21.726979 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:21.726979 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 07:21:21.726979 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 07:21:21.726979 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 07:21:21.726979 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 07:21:21.726979 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:21.726979 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:21.726979 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:21.726979 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:21.726979 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:21.726979 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:21.727803 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:21.727803 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:21.727803 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:21.727803 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:21.727803 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:21.727803 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:21.727803 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:21.727803 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:21.727803 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:21.727803 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:21.727803 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 07:21:21.727803 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 07:21:21.727803 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 15 07:21:21.727803 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:21:21.733195 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 456/896] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Sep 15 07:21:21 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:21:21.734659 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acce374f-7070-4891-888a-75745ac5e0e8 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:21.740686 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 441/897] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 15 07:21:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:21:21.742081 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc206b8-040f-48a8-a321-08ff792dec20 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:21.845687 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc206b8-040f-48a8-a321-08ff792dec20 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:21.850408 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acce374f-7070-4891-888a-75745ac5e0e8 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:21.880640 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:21.886864 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8299c4b-2da0-4a14-8aa2-433fbcc3419e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:21.887469 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8299c4b-2da0-4a14-8aa2-433fbcc3419e None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:21.887753 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8299c4b-2da0-4a14-8aa2-433fbcc3419e None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:21.888029 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8299c4b-2da0-4a14-8aa2-433fbcc3419e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:21.888885 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8299c4b-2da0-4a14-8aa2-433fbcc3419e None admin] RBAC: Adding query filter params (role.id=0888721641c54c3684692df3615ea1b0, scope.project.id=4e9987720930473da840579654311bc2, user.id=f9c68c4bd9074d8d9b893755e28c0807) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:21.889151 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8299c4b-2da0-4a14-8aa2-433fbcc3419e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:21.890602 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8299c4b-2da0-4a14-8aa2-433fbcc3419e None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:21.895328 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52a6853-cb45-41fc-a4c3-658b7a901e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:21.896196 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52a6853-cb45-41fc-a4c3-658b7a901e6d None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:21.896673 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52a6853-cb45-41fc-a4c3-658b7a901e6d None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:21.897143 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52a6853-cb45-41fc-a4c3-658b7a901e6d None admin] PATH_INFO: `/v3/services` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:21.898003 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52a6853-cb45-41fc-a4c3-658b7a901e6d None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:21.898506 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52a6853-cb45-41fc-a4c3-658b7a901e6d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:21.900194 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52a6853-cb45-41fc-a4c3-658b7a901e6d None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:21.902830 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 442/898] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 15 07:21:21 2022] GET /identity/v3/role_assignments?user.id=f9c68c4bd9074d8d9b893755e28c0807&scope.project.id=4e9987720930473da840579654311bc2&role.id=0888721641c54c3684692df3615ea1b0 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:21:21.907448 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 457/899] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Sep 15 07:21:21 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:21.913370 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8299c4b-2da0-4a14-8aa2-433fbcc3419e None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:22.030483 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8299c4b-2da0-4a14-8aa2-433fbcc3419e None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:22.069017 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086176a3-281d-4243-853f-8da24c916407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:22.069656 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086176a3-281d-4243-853f-8da24c916407 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:22.069941 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086176a3-281d-4243-853f-8da24c916407 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:22.070213 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086176a3-281d-4243-853f-8da24c916407 None admin] PATH_INFO: `/v3/services` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:22.070851 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086176a3-281d-4243-853f-8da24c916407 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:22.071149 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086176a3-281d-4243-853f-8da24c916407 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:22.072501 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086176a3-281d-4243-853f-8da24c916407 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:22.078220 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 443/900] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Sep 15 07:21:21 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:21:22.084713 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52a6853-cb45-41fc-a4c3-658b7a901e6d None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:22.129217 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:22.196722 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52a6853-cb45-41fc-a4c3-658b7a901e6d None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:22.233426 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734b9d1b-bd0e-49b3-b997-b4a49947b63a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:22.234072 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734b9d1b-bd0e-49b3-b997-b4a49947b63a None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:22.234354 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734b9d1b-bd0e-49b3-b997-b4a49947b63a None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:22.234678 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734b9d1b-bd0e-49b3-b997-b4a49947b63a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:22.235931 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734b9d1b-bd0e-49b3-b997-b4a49947b63a None admin] RBAC: Adding query filter params (interface=public, service_id=3f17ffe9cba64e9dac10db3f5d465c58, region_id=RegionOne) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:22.236240 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734b9d1b-bd0e-49b3-b997-b4a49947b63a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:22.237758 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734b9d1b-bd0e-49b3-b997-b4a49947b63a None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:22.243363 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 458/901] 10.222.0.42 () {56 vars in 1251 bytes} [Thu Sep 15 07:21:22 2022] GET /identity/v3/endpoints?service_id=3f17ffe9cba64e9dac10db3f5d465c58&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:22.248841 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086176a3-281d-4243-853f-8da24c916407 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:22.314350 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:22.357058 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086176a3-281d-4243-853f-8da24c916407 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:22.394292 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671bacc2-eee0-4677-92cb-b4fd865917b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:22.395312 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671bacc2-eee0-4677-92cb-b4fd865917b8 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:22.395837 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671bacc2-eee0-4677-92cb-b4fd865917b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:22.396319 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671bacc2-eee0-4677-92cb-b4fd865917b8 None admin] PATH_INFO: `/v3/services` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:22.397115 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671bacc2-eee0-4677-92cb-b4fd865917b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:22.398692 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671bacc2-eee0-4677-92cb-b4fd865917b8 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:22.405130 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 444/902] 10.222.0.42 () {56 vars in 1087 bytes} [Thu Sep 15 07:21:22 2022] GET /identity/v3/services => generated 1268 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:21:22.933391 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bf6f53-36b3-4d61-b1d3-9ea2241adb19 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:22.933965 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bf6f53-36b3-4d61-b1d3-9ea2241adb19 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:22.933965 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bf6f53-36b3-4d61-b1d3-9ea2241adb19 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:22.935560 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 459/903] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:21:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:21:22.941528 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f02458-b066-4517-ae82-b48ee9db3795 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:22.942003 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f02458-b066-4517-ae82-b48ee9db3795 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:22.942201 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f02458-b066-4517-ae82-b48ee9db3795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:22.965233 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-13f02458-b066-4517-ae82-b48ee9db3795 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:21:22.966533 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-13f02458-b066-4517-ae82-b48ee9db3795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:21:23.044257 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 445/904] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:21:22 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 07:21:23.119650 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d5fcfd-38ac-4a93-9a0c-031337a65fd1 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:23.120089 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d5fcfd-38ac-4a93-9a0c-031337a65fd1 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:23.120201 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d5fcfd-38ac-4a93-9a0c-031337a65fd1 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:23.121807 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 460/905] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:21:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:21:23.127350 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a501db3d-aac2-4a2f-9a6e-9c9d0d2889e9 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:23.127635 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a501db3d-aac2-4a2f-9a6e-9c9d0d2889e9 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:23.127904 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a501db3d-aac2-4a2f-9a6e-9c9d0d2889e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:23.147126 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-a501db3d-aac2-4a2f-9a6e-9c9d0d2889e9 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:21:23.148317 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-a501db3d-aac2-4a2f-9a6e-9c9d0d2889e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:21:23.199265 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:23.220176 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 446/906] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:21:23 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 07:21:23.310889 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d5fcfd-38ac-4a93-9a0c-031337a65fd1 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:23.424684 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d5fcfd-38ac-4a93-9a0c-031337a65fd1 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:23.426773 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:23.462559 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dbb4c0-dde0-4362-a497-182a777cefb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:23.463488 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dbb4c0-dde0-4362-a497-182a777cefb6 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:23.464021 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dbb4c0-dde0-4362-a497-182a777cefb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:23.464551 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dbb4c0-dde0-4362-a497-182a777cefb6 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:23.465328 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dbb4c0-dde0-4362-a497-182a777cefb6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:23.466980 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dbb4c0-dde0-4362-a497-182a777cefb6 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:23.476130 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-f5dbb4c0-dde0-4362-a497-182a777cefb6 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 15 07:21:23.476130 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:23.476130 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 07:21:23.476130 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 07:21:23.476130 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 07:21:23.476130 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 07:21:23.476130 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 07:21:23.476130 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 07:21:23.476130 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 15 07:21:23.476130 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:21:23.476130 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 07:21:23.476130 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:21:23.476130 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:23.476130 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:23.476130 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:23.476130 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:23.477034 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:23.477034 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:23.477034 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:23.477034 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:23.477034 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:23.477034 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:23.477034 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:23.477034 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 07:21:23.477034 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 07:21:23.477034 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 07:21:23.477034 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 07:21:23.477034 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:23.477034 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:23.477034 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:23.477034 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:23.477034 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:23.477034 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:23.477920 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:23.477920 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:23.477920 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:23.477920 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:23.477920 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:23.477920 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:23.477920 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:23.477920 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:23.477920 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:23.477920 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:23.477920 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 07:21:23.477920 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 07:21:23.477920 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 15 07:21:23.477920 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:21:23.480589 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 461/907] 10.222.0.42 () {56 vars in 1132 bytes} [Thu Sep 15 07:21:23 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:21:23.486302 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a501db3d-aac2-4a2f-9a6e-9c9d0d2889e9 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:23.524023 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dbb4c0-dde0-4362-a497-182a777cefb6 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:23.591303 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a501db3d-aac2-4a2f-9a6e-9c9d0d2889e9 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:23.627511 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4f59a1-63f2-4875-9eb4-23f91074ac92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:23.628131 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4f59a1-63f2-4875-9eb4-23f91074ac92 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:23.628415 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4f59a1-63f2-4875-9eb4-23f91074ac92 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:23.628690 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4f59a1-63f2-4875-9eb4-23f91074ac92 None admin] PATH_INFO: `/v3/services` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:23.629326 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4f59a1-63f2-4875-9eb4-23f91074ac92 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:23.629658 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4f59a1-63f2-4875-9eb4-23f91074ac92 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:23.631290 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4f59a1-63f2-4875-9eb4-23f91074ac92 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:23.640834 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 447/908] 10.222.0.42 () {56 vars in 1126 bytes} [Thu Sep 15 07:21:23 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:23.642582 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4f59a1-63f2-4875-9eb4-23f91074ac92 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:23.648703 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dbb4c0-dde0-4362-a497-182a777cefb6 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:23.692029 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8603758-7c11-4cd4-9387-d6805d854213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:23.692981 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8603758-7c11-4cd4-9387-d6805d854213 None admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:23.693263 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8603758-7c11-4cd4-9387-d6805d854213 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:23.693816 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8603758-7c11-4cd4-9387-d6805d854213 None admin] PATH_INFO: `/v3/services` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:23.694474 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8603758-7c11-4cd4-9387-d6805d854213 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:23.696355 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8603758-7c11-4cd4-9387-d6805d854213 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:23.698900 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:23.751001 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4f59a1-63f2-4875-9eb4-23f91074ac92 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:23.753808 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 462/909] 10.222.0.42 () {60 vars in 1141 bytes} [Thu Sep 15 07:21:23 2022] POST /identity/v3/services => generated 241 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:21:23.805514 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41ea988-1e55-4406-8958-36f4cbd6dd5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:23.806171 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41ea988-1e55-4406-8958-36f4cbd6dd5d None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:23.806453 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41ea988-1e55-4406-8958-36f4cbd6dd5d None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:23.806731 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41ea988-1e55-4406-8958-36f4cbd6dd5d None admin] PATH_INFO: `/v3/services` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:23.807387 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41ea988-1e55-4406-8958-36f4cbd6dd5d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:23.807705 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41ea988-1e55-4406-8958-36f4cbd6dd5d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:23.809278 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41ea988-1e55-4406-8958-36f4cbd6dd5d None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:23.815842 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 448/910] 10.222.0.42 () {56 vars in 1126 bytes} [Thu Sep 15 07:21:23 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:21:23.821996 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8603758-7c11-4cd4-9387-d6805d854213 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:23.860760 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:24.003680 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8603758-7c11-4cd4-9387-d6805d854213 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:24.039948 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9846e460-2c89-4804-b672-07a0b84eec8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:24.040564 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9846e460-2c89-4804-b672-07a0b84eec8a None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:24.040841 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9846e460-2c89-4804-b672-07a0b84eec8a None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:24.041114 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9846e460-2c89-4804-b672-07a0b84eec8a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:24.041854 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9846e460-2c89-4804-b672-07a0b84eec8a None admin] RBAC: Adding query filter params (interface=public, service_id=f9508c29acfa48a79ed06e7cf9f7c5ce, region_id=RegionOne) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:24.042164 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9846e460-2c89-4804-b672-07a0b84eec8a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:24.043533 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9846e460-2c89-4804-b672-07a0b84eec8a None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:24.049105 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 463/911] 10.222.0.42 () {56 vars in 1251 bytes} [Thu Sep 15 07:21:23 2022] GET /identity/v3/endpoints?service_id=f9508c29acfa48a79ed06e7cf9f7c5ce&interface=public®ion_id=RegionOne => generated 194 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:24.054469 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41ea988-1e55-4406-8958-36f4cbd6dd5d None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:24.179990 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41ea988-1e55-4406-8958-36f4cbd6dd5d None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:24.189396 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:24.210645 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c3d4af-9359-42bb-af9c-81cace65f0cd None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:24.213499 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c3d4af-9359-42bb-af9c-81cace65f0cd None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:24.213499 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c3d4af-9359-42bb-af9c-81cace65f0cd None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:24.213499 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 464/912] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:21:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 07:21:24.217951 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf4f655-646c-490d-b9eb-1f75f6da767d None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:24.218241 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf4f655-646c-490d-b9eb-1f75f6da767d None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:24.218409 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82807ab-c69f-425f-acf6-fd30828cd477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:24.218554 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf4f655-646c-490d-b9eb-1f75f6da767d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:24.219082 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82807ab-c69f-425f-acf6-fd30828cd477 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:24.220096 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82807ab-c69f-425f-acf6-fd30828cd477 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:24.220368 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82807ab-c69f-425f-acf6-fd30828cd477 None admin] PATH_INFO: `/v3/services` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:24.220978 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82807ab-c69f-425f-acf6-fd30828cd477 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:24.222851 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82807ab-c69f-425f-acf6-fd30828cd477 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:24.230901 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 449/913] 10.222.0.42 () {56 vars in 1087 bytes} [Thu Sep 15 07:21:24 2022] GET /identity/v3/services => generated 1497 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:21:24.262705 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-cbf4f655-646c-490d-b9eb-1f75f6da767d None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:21:24.263842 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-cbf4f655-646c-490d-b9eb-1f75f6da767d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:21:24.385714 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 465/914] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:21:24 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 07:21:24.672043 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82807ab-c69f-425f-acf6-fd30828cd477 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:24.771759 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b14bb9-302e-4deb-a67d-8e60638ecb4a None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:24.772111 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b14bb9-302e-4deb-a67d-8e60638ecb4a None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:24.772320 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b14bb9-302e-4deb-a67d-8e60638ecb4a None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:24.773934 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 466/915] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:21:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:21:24.779007 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff4d6b8-f734-40d0-92ff-7f6f5f1b0420 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:24.779305 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff4d6b8-f734-40d0-92ff-7f6f5f1b0420 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:24.779565 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff4d6b8-f734-40d0-92ff-7f6f5f1b0420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:24.779785 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82807ab-c69f-425f-acf6-fd30828cd477 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:24.798887 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-6ff4d6b8-f734-40d0-92ff-7f6f5f1b0420 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:21:24.800079 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-6ff4d6b8-f734-40d0-92ff-7f6f5f1b0420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:21:24.815935 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:24.819580 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34555a3-7b18-4d47-bf08-f4cd7d79a2bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:24.820219 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34555a3-7b18-4d47-bf08-f4cd7d79a2bb None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:24.820497 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34555a3-7b18-4d47-bf08-f4cd7d79a2bb None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:24.820771 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34555a3-7b18-4d47-bf08-f4cd7d79a2bb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:24.831505 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34555a3-7b18-4d47-bf08-f4cd7d79a2bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:21:24.836886 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-e34555a3-7b18-4d47-bf08-f4cd7d79a2bb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 15 07:21:24.836886 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:24.836886 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:24.836886 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:24.836886 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:24.836886 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:24.836886 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:24.836886 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:24.836886 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:24.836886 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:24.836886 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:24.836886 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:24.836886 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:21:24.836886 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:21:24.836886 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:21:24.836886 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:21:24.837995 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:21:24.837995 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:21:24.837995 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:21:24.837995 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:21:24.837995 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:24.837995 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:24.837995 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:24.837995 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:24.837995 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:24.837995 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:24.837995 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:24.837995 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:24.837995 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:24.837995 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:24.837995 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:24.837995 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:24.838831 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:24.838831 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:24.838831 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:24.838831 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:24.838831 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:21:24.838831 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:21:24.838831 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:21:24.838831 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:21:24.838831 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:21:24.838831 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:21:24.838831 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 15 07:21:24.838831 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:21:24.838831 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 450/916] 10.222.0.42 () {56 vars in 1120 bytes} [Thu Sep 15 07:21:24 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:21:24.845724 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34555a3-7b18-4d47-bf08-f4cd7d79a2bb None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:24.879104 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:24.880171 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 467/917] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:21:24 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 07:21:24.979766 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34555a3-7b18-4d47-bf08-f4cd7d79a2bb None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:25.021829 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6da2869-f1a9-4da7-9a1b-6cc56b307d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:25.022445 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6da2869-f1a9-4da7-9a1b-6cc56b307d47 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:25.022729 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6da2869-f1a9-4da7-9a1b-6cc56b307d47 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:25.023007 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6da2869-f1a9-4da7-9a1b-6cc56b307d47 None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:25.023678 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6da2869-f1a9-4da7-9a1b-6cc56b307d47 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:25.023983 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6da2869-f1a9-4da7-9a1b-6cc56b307d47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:25.025356 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6da2869-f1a9-4da7-9a1b-6cc56b307d47 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:25.035023 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 451/918] 10.222.0.42 () {56 vars in 1115 bytes} [Thu Sep 15 07:21:24 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:21:25.040600 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff4d6b8-f734-40d0-92ff-7f6f5f1b0420 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:25.145611 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff4d6b8-f734-40d0-92ff-7f6f5f1b0420 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:25.166207 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6da2869-f1a9-4da7-9a1b-6cc56b307d47 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:25.183606 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1dff25-df18-48d1-ab1d-c0417f27e035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:25.184319 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1dff25-df18-48d1-ab1d-c0417f27e035 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:25.184633 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1dff25-df18-48d1-ab1d-c0417f27e035 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:25.184981 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1dff25-df18-48d1-ab1d-c0417f27e035 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:25.197286 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1dff25-df18-48d1-ab1d-c0417f27e035 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:25.198911 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1dff25-df18-48d1-ab1d-c0417f27e035 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:25.216857 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-df1dff25-df18-48d1-ab1d-c0417f27e035 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:21:25.216857 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:25.216857 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:25.216857 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:25.216857 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:25.216857 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:25.216857 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:25.216857 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:25.216857 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:25.216857 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:25.216857 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:25.216857 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:25.216857 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 07:21:25.216857 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 07:21:25.216857 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 07:21:25.216857 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 07:21:25.218785 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:25.218785 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:25.218785 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:25.218785 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:25.218785 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:25.218785 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:25.218785 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:25.218785 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:25.218785 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:25.218785 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:25.218785 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:25.218785 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:25.218785 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:25.218785 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:25.218785 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:25.218785 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:25.218785 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 07:21:25.219721 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 07:21:25.219721 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:21:25.219721 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:21:25.219721 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 468/919] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:21:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:21:25.224445 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1dff25-df18-48d1-ab1d-c0417f27e035 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:25.227572 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:25.277953 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:25.288815 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6da2869-f1a9-4da7-9a1b-6cc56b307d47 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:25.326696 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bbc953-ca8f-4c58-bd3f-03dd861313a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:25.327634 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bbc953-ca8f-4c58-bd3f-03dd861313a6 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:25.328122 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bbc953-ca8f-4c58-bd3f-03dd861313a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:25.328623 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bbc953-ca8f-4c58-bd3f-03dd861313a6 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:25.330193 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bbc953-ca8f-4c58-bd3f-03dd861313a6 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:25.331708 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bbc953-ca8f-4c58-bd3f-03dd861313a6 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:25.341301 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-04bbc953-ca8f-4c58-bd3f-03dd861313a6 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 15 07:21:25.341301 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:25.341301 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 07:21:25.341301 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 07:21:25.341301 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 07:21:25.341301 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 07:21:25.341301 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 07:21:25.341301 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 07:21:25.341301 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 15 07:21:25.341301 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:21:25.341301 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 07:21:25.341301 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:21:25.341301 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:25.341301 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:25.341301 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:25.341301 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:25.341301 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:25.342518 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:25.342518 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:25.342518 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:25.342518 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:25.342518 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:25.342518 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:25.342518 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 07:21:25.342518 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 07:21:25.342518 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 07:21:25.342518 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 07:21:25.342518 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:25.342518 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:25.342518 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:25.342518 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:25.342518 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:25.342518 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:25.343374 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:25.343374 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:25.343374 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:25.343374 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:25.343374 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:25.343374 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:25.343374 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:25.343374 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:25.343374 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:25.343374 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:25.343374 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 07:21:25.343374 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 07:21:25.343374 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 15 07:21:25.343374 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:21:25.345186 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1dff25-df18-48d1-ab1d-c0417f27e035 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:25.351184 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 452/920] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Sep 15 07:21:25 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 07:21:25.352199 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bbc953-ca8f-4c58-bd3f-03dd861313a6 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:25.381104 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595c5851-91c0-4e20-85ae-fa91d1f4fd1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:25.382039 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595c5851-91c0-4e20-85ae-fa91d1f4fd1c None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:25.382399 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595c5851-91c0-4e20-85ae-fa91d1f4fd1c None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:25.382777 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595c5851-91c0-4e20-85ae-fa91d1f4fd1c None admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:25.383494 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595c5851-91c0-4e20-85ae-fa91d1f4fd1c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:25.383868 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595c5851-91c0-4e20-85ae-fa91d1f4fd1c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:25.385307 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595c5851-91c0-4e20-85ae-fa91d1f4fd1c None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:25.403909 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 469/921] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 15 07:21:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:21:25.404930 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595c5851-91c0-4e20-85ae-fa91d1f4fd1c None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:25.464754 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bbc953-ca8f-4c58-bd3f-03dd861313a6 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:25.470054 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:25.503069 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517b33e8-914f-46ad-be03-fa81001a273c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:25.503939 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517b33e8-914f-46ad-be03-fa81001a273c None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:25.505120 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517b33e8-914f-46ad-be03-fa81001a273c None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:25.505876 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517b33e8-914f-46ad-be03-fa81001a273c None admin] PATH_INFO: `/v3/services` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:25.506646 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517b33e8-914f-46ad-be03-fa81001a273c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:25.507190 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517b33e8-914f-46ad-be03-fa81001a273c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:25.508787 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517b33e8-914f-46ad-be03-fa81001a273c None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:25.511110 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595c5851-91c0-4e20-85ae-fa91d1f4fd1c None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:25.519044 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 453/922] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Sep 15 07:21:25 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:25.520436 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517b33e8-914f-46ad-be03-fa81001a273c None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:25.545359 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:25.556176 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83270a0-445b-4d2f-9c46-c381b3708d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:25.557094 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83270a0-445b-4d2f-9c46-c381b3708d6c None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:25.557234 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83270a0-445b-4d2f-9c46-c381b3708d6c None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:25.557568 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83270a0-445b-4d2f-9c46-c381b3708d6c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:25.566117 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83270a0-445b-4d2f-9c46-c381b3708d6c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:25.567745 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83270a0-445b-4d2f-9c46-c381b3708d6c None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:25.584386 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-d83270a0-445b-4d2f-9c46-c381b3708d6c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 15 07:21:25.584386 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:25.584386 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:25.584386 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:25.584386 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:25.584386 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:25.584386 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:25.584386 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:25.584386 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:25.584386 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:25.584386 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:25.584386 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:25.584386 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 07:21:25.584386 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 07:21:25.584386 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 07:21:25.584386 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 07:21:25.585552 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:25.585552 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:25.585552 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 07:21:25.585552 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 07:21:25.585552 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 07:21:25.585552 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 07:21:25.585552 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:25.585552 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:25.585552 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:25.585552 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:25.585552 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:25.585552 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:25.585552 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:25.585552 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:25.585552 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:25.585552 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:25.585552 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:25.585552 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:25.586663 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:25.586663 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:25.586663 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 07:21:25.586663 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 07:21:25.586663 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 07:21:25.586663 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 07:21:25.586663 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 07:21:25.586663 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 07:21:25.586663 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 15 07:21:25.586663 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:21:25.591730 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 470/923] 10.222.0.42 () {56 vars in 1117 bytes} [Thu Sep 15 07:21:25 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:21:25.592649 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83270a0-445b-4d2f-9c46-c381b3708d6c None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:25.639461 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517b33e8-914f-46ad-be03-fa81001a273c None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:25.677493 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bc7f0b-a126-4891-ba6a-525e52368193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:25.678450 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bc7f0b-a126-4891-ba6a-525e52368193 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:25.678908 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bc7f0b-a126-4891-ba6a-525e52368193 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:25.679438 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bc7f0b-a126-4891-ba6a-525e52368193 None admin] PATH_INFO: `/v3/services` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:25.680362 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bc7f0b-a126-4891-ba6a-525e52368193 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:25.680855 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bc7f0b-a126-4891-ba6a-525e52368193 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:25.682399 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bc7f0b-a126-4891-ba6a-525e52368193 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:25.693330 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 454/924] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Sep 15 07:21:25 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:21:25.694506 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bc7f0b-a126-4891-ba6a-525e52368193 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:25.699938 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83270a0-445b-4d2f-9c46-c381b3708d6c None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:25.724220 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:25.736919 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c614a909-674e-429e-9efb-43ffd4d32179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:25.737536 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c614a909-674e-429e-9efb-43ffd4d32179 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:25.737855 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c614a909-674e-429e-9efb-43ffd4d32179 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:25.738638 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c614a909-674e-429e-9efb-43ffd4d32179 None admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:25.738917 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c614a909-674e-429e-9efb-43ffd4d32179 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:25.739229 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c614a909-674e-429e-9efb-43ffd4d32179 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:25.741081 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c614a909-674e-429e-9efb-43ffd4d32179 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:25.765529 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 471/925] 10.222.0.42 () {56 vars in 1149 bytes} [Thu Sep 15 07:21:25 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:25.766733 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c614a909-674e-429e-9efb-43ffd4d32179 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:25.789625 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:25.807383 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bc7f0b-a126-4891-ba6a-525e52368193 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:25.846487 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25d7e75-1eb2-46f4-8afe-21142a44fd6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:25.847398 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25d7e75-1eb2-46f4-8afe-21142a44fd6d None admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:25.847969 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25d7e75-1eb2-46f4-8afe-21142a44fd6d None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:25.848438 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25d7e75-1eb2-46f4-8afe-21142a44fd6d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:25.849500 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25d7e75-1eb2-46f4-8afe-21142a44fd6d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:25.851053 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25d7e75-1eb2-46f4-8afe-21142a44fd6d None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:25.881916 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c614a909-674e-429e-9efb-43ffd4d32179 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:25.921093 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a64528-26d3-40f7-b39a-12d03053ce56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:25.921822 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a64528-26d3-40f7-b39a-12d03053ce56 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:25.922105 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a64528-26d3-40f7-b39a-12d03053ce56 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:25.922381 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a64528-26d3-40f7-b39a-12d03053ce56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:25.942043 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 455/926] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:21:25 2022] POST /identity/v3/endpoints => generated 346 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:21:25.953039 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a64528-26d3-40f7-b39a-12d03053ce56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:25.953039 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a64528-26d3-40f7-b39a-12d03053ce56 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:25.970829 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-e5a64528-26d3-40f7-b39a-12d03053ce56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:21:25.970829 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:25.970829 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:25.970829 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:25.970829 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:25.970829 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:25.970829 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:25.970829 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:25.970829 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:25.970829 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:25.970829 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:25.970829 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:25.970829 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 07:21:25.970829 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 07:21:25.970829 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 07:21:25.970829 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 07:21:25.971867 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:25.971867 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:25.971867 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:25.971867 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:25.971867 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:25.971867 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:25.971867 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:25.971867 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:25.971867 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:25.971867 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:25.971867 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:25.971867 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:25.971867 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:25.971867 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:25.971867 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:25.971867 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:25.971867 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 07:21:25.972771 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 07:21:25.972771 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:21:25.972771 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:21:25.974798 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 472/927] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:21:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:21:25.981227 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25d7e75-1eb2-46f4-8afe-21142a44fd6d None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:26.111881 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25d7e75-1eb2-46f4-8afe-21142a44fd6d None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:26.152319 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e901713-2f91-4514-98f0-65ca5eb7496f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:26.153351 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e901713-2f91-4514-98f0-65ca5eb7496f None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:26.153897 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e901713-2f91-4514-98f0-65ca5eb7496f None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:26.154387 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e901713-2f91-4514-98f0-65ca5eb7496f None admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:26.155313 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e901713-2f91-4514-98f0-65ca5eb7496f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:26.155630 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e901713-2f91-4514-98f0-65ca5eb7496f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:26.159467 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e901713-2f91-4514-98f0-65ca5eb7496f None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:26.167405 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:26.181254 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 456/928] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 15 07:21:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:26.189665 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a64528-26d3-40f7-b39a-12d03053ce56 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:26.226058 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:26.283880 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2cee6c-8463-4137-a555-9e930bd07cc3 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:26.284150 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2cee6c-8463-4137-a555-9e930bd07cc3 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:26.284413 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2cee6c-8463-4137-a555-9e930bd07cc3 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:26.286045 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 457/929] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:21:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 07:21:26.291594 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db56193a-70ff-4566-a536-f9281a7a29a1 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:26.291879 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db56193a-70ff-4566-a536-f9281a7a29a1 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:26.292160 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db56193a-70ff-4566-a536-f9281a7a29a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:26.304088 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a64528-26d3-40f7-b39a-12d03053ce56 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:26.311628 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-db56193a-70ff-4566-a536-f9281a7a29a1 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:21:26.314042 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-db56193a-70ff-4566-a536-f9281a7a29a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:21:26.351912 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd35240b-e2cc-434d-9d6d-5118f7bd7caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:26.353086 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd35240b-e2cc-434d-9d6d-5118f7bd7caa None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:26.354420 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd35240b-e2cc-434d-9d6d-5118f7bd7caa None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:26.354908 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd35240b-e2cc-434d-9d6d-5118f7bd7caa None admin] PATH_INFO: `/v3/projects/service` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:26.363770 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd35240b-e2cc-434d-9d6d-5118f7bd7caa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:26.365783 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd35240b-e2cc-434d-9d6d-5118f7bd7caa None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:26.380982 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-fd35240b-e2cc-434d-9d6d-5118f7bd7caa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 07:21:26.380982 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:26.380982 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:26.380982 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:26.380982 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:26.380982 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:26.380982 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:26.380982 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:26.380982 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:26.380982 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:26.380982 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:26.380982 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:26.380982 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 07:21:26.380982 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 07:21:26.380982 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 07:21:26.381976 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 07:21:26.381976 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:26.381976 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:26.381976 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:26.381976 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:26.381976 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:26.381976 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:26.381976 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:26.381976 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:26.381976 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:26.381976 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:26.381976 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:26.381976 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:26.381976 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:26.381976 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:26.381976 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:26.381976 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:26.382847 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 07:21:26.382847 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 07:21:26.382847 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 07:21:26.382847 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 07:21:26.382847 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 07:21:26.382847 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 07:21:26.382847 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 07:21:26.382847 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:21:26.387381 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 473/930] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 15 07:21:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:21:26.388362 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd35240b-e2cc-434d-9d6d-5118f7bd7caa None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:26.392998 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 458/931] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:21:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 07:21:26.492903 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:26.505281 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd35240b-e2cc-434d-9d6d-5118f7bd7caa None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:26.517360 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec124f4-64a7-43fe-846f-7c63b53c4690 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:26.517698 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec124f4-64a7-43fe-846f-7c63b53c4690 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:26.518005 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec124f4-64a7-43fe-846f-7c63b53c4690 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:26.521060 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 459/932] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:21:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 07:21:26.524947 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b78202-6408-4ef7-ae6c-2407a53299c1 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:26.525231 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b78202-6408-4ef7-ae6c-2407a53299c1 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:26.525513 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b78202-6408-4ef7-ae6c-2407a53299c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:26.548683 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-91b78202-6408-4ef7-ae6c-2407a53299c1 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:21:26.549888 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-91b78202-6408-4ef7-ae6c-2407a53299c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:21:26.551379 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d657c0c-3b31-4e7c-b407-cc273887a5ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:26.552421 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d657c0c-3b31-4e7c-b407-cc273887a5ab None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:26.553707 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d657c0c-3b31-4e7c-b407-cc273887a5ab None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:26.554085 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d657c0c-3b31-4e7c-b407-cc273887a5ab None admin] PATH_INFO: `/v3/projects` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:26.554932 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d657c0c-3b31-4e7c-b407-cc273887a5ab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:26.555331 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d657c0c-3b31-4e7c-b407-cc273887a5ab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:26.556905 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d657c0c-3b31-4e7c-b407-cc273887a5ab None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:26.575452 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 474/933] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 15 07:21:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:21:26.581058 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d657c0c-3b31-4e7c-b407-cc273887a5ab None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:26.638724 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 460/934] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:21:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 07:21:26.695401 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d657c0c-3b31-4e7c-b407-cc273887a5ab None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:26.706435 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b78202-6408-4ef7-ae6c-2407a53299c1 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:26.734847 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93455bc-08b5-4945-b7d6-80f546be00eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:26.736371 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93455bc-08b5-4945-b7d6-80f546be00eb None admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:26.736841 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93455bc-08b5-4945-b7d6-80f546be00eb None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:26.737352 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93455bc-08b5-4945-b7d6-80f546be00eb None admin] PATH_INFO: `/v3/projects/4e9987720930473da840579654311bc2/users/f9c68c4bd9074d8d9b893755e28c0807/roles/0888721641c54c3684692df3615ea1b0` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:26.739897 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93455bc-08b5-4945-b7d6-80f546be00eb None admin] RBAC: Authorizing `identity:create_grant(project_id=4e9987720930473da840579654311bc2, user_id=f9c68c4bd9074d8d9b893755e28c0807, role_id=0888721641c54c3684692df3615ea1b0)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:26.741834 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93455bc-08b5-4945-b7d6-80f546be00eb None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:26.768689 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:26.813009 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b78202-6408-4ef7-ae6c-2407a53299c1 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:26.849028 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cde4f0f-10dc-4bcf-ba02-d4922342b2a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:26.849696 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cde4f0f-10dc-4bcf-ba02-d4922342b2a6 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:26.849975 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cde4f0f-10dc-4bcf-ba02-d4922342b2a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:26.850243 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cde4f0f-10dc-4bcf-ba02-d4922342b2a6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:26.850821 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cde4f0f-10dc-4bcf-ba02-d4922342b2a6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:26.852392 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cde4f0f-10dc-4bcf-ba02-d4922342b2a6 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:26.860995 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-6cde4f0f-10dc-4bcf-ba02-d4922342b2a6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 07:21:26.860995 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:26.860995 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 07:21:26.860995 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 07:21:26.860995 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 07:21:26.860995 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 07:21:26.860995 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 07:21:26.860995 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 07:21:26.860995 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 07:21:26.860995 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:21:26.860995 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 07:21:26.860995 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:21:26.860995 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:26.860995 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:26.860995 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:26.860995 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:26.860995 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:26.862206 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:26.862206 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:26.862206 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:26.862206 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:26.862206 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:26.862206 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:26.862206 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 07:21:26.862206 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 07:21:26.862206 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 07:21:26.862206 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 07:21:26.862206 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:26.862206 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:26.862206 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:26.862206 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:26.862206 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:26.862206 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:26.863589 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:26.863589 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:26.863589 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:26.863589 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:26.863589 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:26.863589 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:26.863589 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:26.863589 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:26.863589 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:26.863589 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:26.863589 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 07:21:26.863589 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 07:21:26.863589 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 07:21:26.863589 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:21:26.865918 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 461/935] 10.222.0.42 () {56 vars in 1126 bytes} [Thu Sep 15 07:21:26 2022] GET /identity/v3/services/object-store => generated 93 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 07:21:26.867849 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cde4f0f-10dc-4bcf-ba02-d4922342b2a6 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:26.882490 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 475/936] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Sep 15 07:21:26 2022] PUT /identity/v3/projects/4e9987720930473da840579654311bc2/users/f9c68c4bd9074d8d9b893755e28c0807/roles/0888721641c54c3684692df3615ea1b0 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 07:21:26.943174 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93455bc-08b5-4945-b7d6-80f546be00eb None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:26.960129 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:27.009829 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cde4f0f-10dc-4bcf-ba02-d4922342b2a6 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:27.037471 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:27.056263 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b653a0-d343-48fc-9902-598db317b1ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:27.056891 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b653a0-d343-48fc-9902-598db317b1ba None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:27.057172 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b653a0-d343-48fc-9902-598db317b1ba None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:27.057493 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b653a0-d343-48fc-9902-598db317b1ba None admin] PATH_INFO: `/v3/services` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:27.058353 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b653a0-d343-48fc-9902-598db317b1ba None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:27.058659 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b653a0-d343-48fc-9902-598db317b1ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:27.060829 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b653a0-d343-48fc-9902-598db317b1ba None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:27.066215 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93455bc-08b5-4945-b7d6-80f546be00eb None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:27.068585 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 462/937] 10.222.0.42 () {56 vars in 1122 bytes} [Thu Sep 15 07:21:26 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:21:27.072967 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b653a0-d343-48fc-9902-598db317b1ba None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:27.115618 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e36043-adf6-4a48-a03c-502f9ceda2d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:27.116392 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e36043-adf6-4a48-a03c-502f9ceda2d6 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:27.116684 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e36043-adf6-4a48-a03c-502f9ceda2d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:27.116957 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e36043-adf6-4a48-a03c-502f9ceda2d6 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:27.118356 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e36043-adf6-4a48-a03c-502f9ceda2d6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:27.119728 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e36043-adf6-4a48-a03c-502f9ceda2d6 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:27.132330 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-91e36043-adf6-4a48-a03c-502f9ceda2d6 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 15 07:21:27.132330 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:27.132330 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 07:21:27.132330 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 07:21:27.132330 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 07:21:27.132330 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 07:21:27.132330 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 07:21:27.132330 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 07:21:27.132330 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 15 07:21:27.132330 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:21:27.132330 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 07:21:27.132330 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:21:27.132330 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:27.132330 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:27.132330 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:27.132330 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:27.133488 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:27.133488 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:27.133488 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:27.133488 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:27.133488 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:27.133488 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:27.133488 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:27.133488 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 07:21:27.133488 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 07:21:27.133488 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 07:21:27.133488 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 07:21:27.133488 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:27.133488 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:27.133488 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:27.133488 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:27.133488 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:27.133488 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:27.134654 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:27.134654 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:27.134654 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:27.134654 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:27.134654 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:27.134654 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:27.134654 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:27.134654 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:27.134654 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:27.134654 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:27.134654 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 07:21:27.134654 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 07:21:27.134654 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 15 07:21:27.134654 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:21:27.134654 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 476/938] 10.222.0.42 () {56 vars in 1132 bytes} [Thu Sep 15 07:21:26 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 07:21:27.141746 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e36043-adf6-4a48-a03c-502f9ceda2d6 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:27.207680 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b653a0-d343-48fc-9902-598db317b1ba None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:27.245729 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebea6e0-2388-4f35-b180-7d8d3adee2a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:27.246389 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebea6e0-2388-4f35-b180-7d8d3adee2a0 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:27.246674 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebea6e0-2388-4f35-b180-7d8d3adee2a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:27.246993 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebea6e0-2388-4f35-b180-7d8d3adee2a0 None admin] PATH_INFO: `/v3/services` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:27.247627 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebea6e0-2388-4f35-b180-7d8d3adee2a0 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:27.247931 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebea6e0-2388-4f35-b180-7d8d3adee2a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:27.249280 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebea6e0-2388-4f35-b180-7d8d3adee2a0 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:27.255346 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 463/939] 10.222.0.42 () {56 vars in 1122 bytes} [Thu Sep 15 07:21:27 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:27.256106 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:27.261200 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebea6e0-2388-4f35-b180-7d8d3adee2a0 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:27.288545 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e36043-adf6-4a48-a03c-502f9ceda2d6 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:27.318516 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:27.325264 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca70b4d-37b8-4a9a-b8bd-42e2bf993805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:27.326005 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca70b4d-37b8-4a9a-b8bd-42e2bf993805 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:27.326381 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca70b4d-37b8-4a9a-b8bd-42e2bf993805 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:27.326747 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca70b4d-37b8-4a9a-b8bd-42e2bf993805 None admin] PATH_INFO: `/v3/services` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:27.327466 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca70b4d-37b8-4a9a-b8bd-42e2bf993805 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:27.327880 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca70b4d-37b8-4a9a-b8bd-42e2bf993805 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:27.330036 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca70b4d-37b8-4a9a-b8bd-42e2bf993805 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:27.339244 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 477/940] 10.222.0.42 () {56 vars in 1126 bytes} [Thu Sep 15 07:21:27 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:27.340780 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca70b4d-37b8-4a9a-b8bd-42e2bf993805 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:27.377510 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebea6e0-2388-4f35-b180-7d8d3adee2a0 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:27.416549 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77441956-fc14-4191-8e56-383ca3294b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:27.417667 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77441956-fc14-4191-8e56-383ca3294b8f None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:27.418156 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77441956-fc14-4191-8e56-383ca3294b8f None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:27.418628 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77441956-fc14-4191-8e56-383ca3294b8f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:27.419551 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77441956-fc14-4191-8e56-383ca3294b8f None admin] RBAC: Adding query filter params (interface=public, service_id=1510b5edd14248b4ac75514172d6ea2b, region_id=RegionOne) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:27.420087 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77441956-fc14-4191-8e56-383ca3294b8f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:27.421665 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77441956-fc14-4191-8e56-383ca3294b8f None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:27.431199 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 464/941] 10.222.0.42 () {56 vars in 1251 bytes} [Thu Sep 15 07:21:27 2022] GET /identity/v3/endpoints?service_id=1510b5edd14248b4ac75514172d6ea2b&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:27.436603 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77441956-fc14-4191-8e56-383ca3294b8f None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:27.458206 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca70b4d-37b8-4a9a-b8bd-42e2bf993805 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:27.494833 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa3c2aa-3025-493a-93a3-1a7ced578675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:27.495818 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa3c2aa-3025-493a-93a3-1a7ced578675 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:27.496464 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa3c2aa-3025-493a-93a3-1a7ced578675 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:27.497026 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa3c2aa-3025-493a-93a3-1a7ced578675 None admin] PATH_INFO: `/v3/services` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:27.497869 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa3c2aa-3025-493a-93a3-1a7ced578675 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:27.498401 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa3c2aa-3025-493a-93a3-1a7ced578675 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:27.500135 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa3c2aa-3025-493a-93a3-1a7ced578675 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:27.510310 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 478/942] 10.222.0.42 () {56 vars in 1126 bytes} [Thu Sep 15 07:21:27 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:27.511570 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa3c2aa-3025-493a-93a3-1a7ced578675 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:27.515539 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:27.543173 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77441956-fc14-4191-8e56-383ca3294b8f None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:27.557162 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:27.579657 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f22c3a5-36af-4807-8417-f2f528a1ad13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:27.580575 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f22c3a5-36af-4807-8417-f2f528a1ad13 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:27.580874 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f22c3a5-36af-4807-8417-f2f528a1ad13 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:27.581128 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f22c3a5-36af-4807-8417-f2f528a1ad13 None admin] PATH_INFO: `/v3/services` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:27.581768 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f22c3a5-36af-4807-8417-f2f528a1ad13 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:27.583177 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f22c3a5-36af-4807-8417-f2f528a1ad13 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:27.597612 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 465/943] 10.222.0.42 () {56 vars in 1087 bytes} [Thu Sep 15 07:21:27 2022] GET /identity/v3/services => generated 1497 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 07:21:27.638890 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa3c2aa-3025-493a-93a3-1a7ced578675 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:27.677539 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f66ab5-6f99-46c0-8381-7f64c2c2f3c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:27.678198 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f66ab5-6f99-46c0-8381-7f64c2c2f3c4 None admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:27.678479 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f66ab5-6f99-46c0-8381-7f64c2c2f3c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:27.678750 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f66ab5-6f99-46c0-8381-7f64c2c2f3c4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:27.679440 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f66ab5-6f99-46c0-8381-7f64c2c2f3c4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:27.680826 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f66ab5-6f99-46c0-8381-7f64c2c2f3c4 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:27.784092 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 479/944] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:21:27 2022] POST /identity/v3/endpoints => generated 347 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:21:28.274177 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e11f08-ac07-4282-a2c9-c3824c0d3121 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:28.274671 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e11f08-ac07-4282-a2c9-c3824c0d3121 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:28.274671 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e11f08-ac07-4282-a2c9-c3824c0d3121 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:28.276221 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 466/945] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:21:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:21:28.281876 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a6819f-0a8d-49b6-83a5-ec2ee92fb960 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:28.282467 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a6819f-0a8d-49b6-83a5-ec2ee92fb960 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:28.282628 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a6819f-0a8d-49b6-83a5-ec2ee92fb960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:28.301753 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-61a6819f-0a8d-49b6-83a5-ec2ee92fb960 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:21:28.302936 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-61a6819f-0a8d-49b6-83a5-ec2ee92fb960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:21:28.361964 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:28.380452 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 480/946] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:21:28 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 07:21:28.650343 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e11f08-ac07-4282-a2c9-c3824c0d3121 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:28.757918 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e11f08-ac07-4282-a2c9-c3824c0d3121 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:28.793964 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e5fc15-e1b5-4dfa-b0ca-6fe90ec4889b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:28.794666 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e5fc15-e1b5-4dfa-b0ca-6fe90ec4889b None admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:28.794952 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e5fc15-e1b5-4dfa-b0ca-6fe90ec4889b None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:28.795225 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e5fc15-e1b5-4dfa-b0ca-6fe90ec4889b None admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:28.795883 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e5fc15-e1b5-4dfa-b0ca-6fe90ec4889b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:28.797308 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e5fc15-e1b5-4dfa-b0ca-6fe90ec4889b None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:28.800218 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: WARNING keystone.server.flask.common [None req-a3e5fc15-e1b5-4dfa-b0ca-6fe90ec4889b None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 15 07:21:28.848994 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.sql.core [None req-a3e5fc15-e1b5-4dfa-b0ca-6fe90ec4889b None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 15 07:21:28.848994 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 15 07:21:28.848994 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [parameters: {'id': 'f5f027e355cb4bfcbf851d9eda8c3300', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 15 07:21:28.848994 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81597) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 15 07:21:28.856367 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-a3e5fc15-e1b5-4dfa-b0ca-6fe90ec4889b None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f5f027e355cb4bfcbf851d9eda8c3300.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f5f027e355cb4bfcbf851d9eda8c3300. Sep 15 07:21:28.856367 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:28.856367 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 15 07:21:28.856367 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 15 07:21:28.856367 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 15 07:21:28.856367 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 15 07:21:28.856367 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 15 07:21:28.856367 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application result = self._query(query) Sep 15 07:21:28.856367 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 15 07:21:28.856367 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application conn.query(q) Sep 15 07:21:28.856367 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 15 07:21:28.856367 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 15 07:21:28.856367 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 15 07:21:28.857165 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application result.read() Sep 15 07:21:28.857165 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 15 07:21:28.857165 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 15 07:21:28.857165 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 15 07:21:28.857165 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 15 07:21:28.857165 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 15 07:21:28.857165 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 15 07:21:28.857165 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 15 07:21:28.857165 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 15 07:21:28.857165 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 15 07:21:28.857165 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:21:28.857165 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 15 07:21:28.857165 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:21:28.857165 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:28.857165 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 15 07:21:28.857165 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 15 07:21:28.857165 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 15 07:21:28.857165 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 15 07:21:28.858140 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 15 07:21:28.858140 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application next(self.gen) Sep 15 07:21:28.858140 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 15 07:21:28.858140 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application yield resource Sep 15 07:21:28.858140 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 15 07:21:28.858140 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application next(self.gen) Sep 15 07:21:28.858140 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 15 07:21:28.858140 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application self.session.rollback() Sep 15 07:21:28.858140 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 15 07:21:28.858140 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application self.force_reraise() Sep 15 07:21:28.858140 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 15 07:21:28.858140 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise self.value Sep 15 07:21:28.858140 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 15 07:21:28.858140 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 15 07:21:28.858140 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 15 07:21:28.858140 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application session.commit() Sep 15 07:21:28.858140 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 15 07:21:28.859057 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 15 07:21:28.859057 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 15 07:21:28.859057 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application self._prepare_impl() Sep 15 07:21:28.859057 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 15 07:21:28.859057 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application self.session.flush() Sep 15 07:21:28.859057 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 15 07:21:28.859057 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application self._flush(objects) Sep 15 07:21:28.859057 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 15 07:21:28.859057 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 15 07:21:28.859057 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 15 07:21:28.859057 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application compat.raise_( Sep 15 07:21:28.859057 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 15 07:21:28.859057 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception Sep 15 07:21:28.859057 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 15 07:21:28.859057 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application flush_context.execute() Sep 15 07:21:28.859057 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 15 07:21:28.859057 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rec.execute(self) Sep 15 07:21:28.859954 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 15 07:21:28.859954 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 15 07:21:28.859954 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 15 07:21:28.859954 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 15 07:21:28.859954 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 15 07:21:28.859954 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 15 07:21:28.859954 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 15 07:21:28.859954 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 15 07:21:28.859954 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 15 07:21:28.859954 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 15 07:21:28.859954 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 15 07:21:28.859954 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 15 07:21:28.859954 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 15 07:21:28.859954 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 15 07:21:28.859954 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 15 07:21:28.861708 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 15 07:21:28.861708 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 15 07:21:28.861708 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception Sep 15 07:21:28.861708 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 15 07:21:28.861708 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 15 07:21:28.861708 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 15 07:21:28.861708 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 15 07:21:28.861708 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 15 07:21:28.861708 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application result = self._query(query) Sep 15 07:21:28.861708 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 15 07:21:28.861708 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application conn.query(q) Sep 15 07:21:28.861708 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 15 07:21:28.861708 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 15 07:21:28.861708 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 15 07:21:28.861708 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application result.read() Sep 15 07:21:28.861708 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 15 07:21:28.861708 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 15 07:21:28.862589 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 15 07:21:28.862589 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 15 07:21:28.862589 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 15 07:21:28.862589 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 15 07:21:28.862589 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 15 07:21:28.862589 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 15 07:21:28.862589 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 15 07:21:28.862589 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 15 07:21:28.862589 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [parameters: {'id': 'f5f027e355cb4bfcbf851d9eda8c3300', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 15 07:21:28.862589 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 15 07:21:28.862589 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:21:28.862589 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 07:21:28.862589 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:21:28.862589 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:28.862589 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 15 07:21:28.863376 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 15 07:21:28.863376 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 15 07:21:28.863376 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 15 07:21:28.863376 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 15 07:21:28.863376 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:21:28.863376 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 07:21:28.863376 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:21:28.863376 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:28.863376 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:28.863376 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:28.863376 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:28.863376 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:28.863376 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:28.863376 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:28.863376 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:28.863376 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:28.863376 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:28.864292 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:28.864292 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 15 07:21:28.864292 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 15 07:21:28.864292 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:28.864292 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:28.864292 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 15 07:21:28.864292 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 15 07:21:28.864292 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f5f027e355cb4bfcbf851d9eda8c3300. Sep 15 07:21:28.864292 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:21:28.864292 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 467/947] 10.222.0.42 () {60 vars in 1140 bytes} [Thu Sep 15 07:21:28 2022] POST /identity/v3/projects => generated 262 bytes in 208 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:21:28.864847 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a6819f-0a8d-49b6-83a5-ec2ee92fb960 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:28.971711 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a6819f-0a8d-49b6-83a5-ec2ee92fb960 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:29.007316 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cd46c7-0deb-4d51-952f-edc45b17b9f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:29.008274 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cd46c7-0deb-4d51-952f-edc45b17b9f4 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:29.008695 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cd46c7-0deb-4d51-952f-edc45b17b9f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:29.009107 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cd46c7-0deb-4d51-952f-edc45b17b9f4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:29.017360 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cd46c7-0deb-4d51-952f-edc45b17b9f4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:29.019141 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cd46c7-0deb-4d51-952f-edc45b17b9f4 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:29.033520 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-71cd46c7-0deb-4d51-952f-edc45b17b9f4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 07:21:29.033520 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:29.033520 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:29.033520 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:29.033520 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:29.033520 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:29.033520 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:29.033520 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:29.033520 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:29.033520 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:29.033520 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:29.033520 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:29.033520 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 07:21:29.033520 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 07:21:29.033520 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 07:21:29.034665 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 07:21:29.034665 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:29.034665 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:29.034665 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:29.034665 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:29.034665 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:29.034665 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:29.034665 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:29.034665 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:29.034665 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:29.034665 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:29.034665 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:29.034665 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:29.034665 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:29.034665 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:29.034665 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:29.034665 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:29.035776 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 07:21:29.035776 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 07:21:29.035776 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 07:21:29.035776 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 07:21:29.035776 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 07:21:29.035776 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 07:21:29.035776 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 07:21:29.035776 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:21:29.041704 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 481/948] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 15 07:21:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:21:29.044606 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e5fc15-e1b5-4dfa-b0ca-6fe90ec4889b None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:29.081303 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:29.140190 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687a649c-2e06-48ca-8bf2-93b05567d2b1 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:29.140526 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687a649c-2e06-48ca-8bf2-93b05567d2b1 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:29.140798 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687a649c-2e06-48ca-8bf2-93b05567d2b1 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:29.143029 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 482/949] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:21:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 07:21:29.148114 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e645cef-90d6-4df7-86a8-055e8fec1efa None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:29.148396 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e645cef-90d6-4df7-86a8-055e8fec1efa None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:29.148668 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e645cef-90d6-4df7-86a8-055e8fec1efa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:29.155506 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e5fc15-e1b5-4dfa-b0ca-6fe90ec4889b None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:29.167577 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-5e645cef-90d6-4df7-86a8-055e8fec1efa None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:21:29.169413 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-5e645cef-90d6-4df7-86a8-055e8fec1efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:21:29.208368 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804f4cce-ca89-418f-bc1f-29d628a07de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:29.208911 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804f4cce-ca89-418f-bc1f-29d628a07de9 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:29.209205 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804f4cce-ca89-418f-bc1f-29d628a07de9 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:29.209473 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804f4cce-ca89-418f-bc1f-29d628a07de9 None admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:29.210184 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804f4cce-ca89-418f-bc1f-29d628a07de9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:29.210496 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804f4cce-ca89-418f-bc1f-29d628a07de9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:29.212147 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804f4cce-ca89-418f-bc1f-29d628a07de9 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:29.233958 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:29.239707 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 468/950] 10.222.0.42 () {56 vars in 1112 bytes} [Thu Sep 15 07:21:29 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:29.261035 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 483/951] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:21:29 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 07:21:29.546614 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804f4cce-ca89-418f-bc1f-29d628a07de9 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:29.647551 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:29.656262 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804f4cce-ca89-418f-bc1f-29d628a07de9 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:29.693992 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8fa670-027d-42eb-9b31-b0bc3da3090e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:29.694694 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8fa670-027d-42eb-9b31-b0bc3da3090e None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:29.694977 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8fa670-027d-42eb-9b31-b0bc3da3090e None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:29.695248 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8fa670-027d-42eb-9b31-b0bc3da3090e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:29.706818 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8fa670-027d-42eb-9b31-b0bc3da3090e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:21:29.711940 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-9e8fa670-027d-42eb-9b31-b0bc3da3090e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 15 07:21:29.711940 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:29.711940 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:29.711940 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:29.711940 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:29.711940 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:29.711940 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:29.711940 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:29.711940 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:29.711940 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:29.711940 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:29.711940 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:29.711940 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:21:29.711940 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:21:29.711940 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:21:29.711940 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:21:29.712961 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:21:29.712961 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:21:29.712961 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:21:29.712961 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:21:29.712961 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:29.712961 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:29.712961 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:29.712961 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:29.712961 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:29.712961 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:29.712961 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:29.712961 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:29.712961 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:29.712961 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:29.712961 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:29.712961 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:29.713897 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:29.713897 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:29.713897 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:29.713897 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:29.713897 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:21:29.713897 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:21:29.713897 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:21:29.713897 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:21:29.713897 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:21:29.713897 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:21:29.713897 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 15 07:21:29.713897 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:21:29.713897 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 469/952] 10.222.0.42 () {56 vars in 1120 bytes} [Thu Sep 15 07:21:29 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:21:29.719307 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e645cef-90d6-4df7-86a8-055e8fec1efa None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:29.831048 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e645cef-90d6-4df7-86a8-055e8fec1efa None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:29.871520 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffc7f5b-6430-457e-9378-100fe9c58d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:29.872140 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffc7f5b-6430-457e-9378-100fe9c58d76 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:29.872424 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffc7f5b-6430-457e-9378-100fe9c58d76 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:29.872702 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffc7f5b-6430-457e-9378-100fe9c58d76 None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:29.873374 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffc7f5b-6430-457e-9378-100fe9c58d76 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:29.873713 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffc7f5b-6430-457e-9378-100fe9c58d76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:29.875094 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffc7f5b-6430-457e-9378-100fe9c58d76 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:29.884895 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 484/953] 10.222.0.42 () {56 vars in 1115 bytes} [Thu Sep 15 07:21:29 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:29.890484 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8fa670-027d-42eb-9b31-b0bc3da3090e None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:29.928081 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4da261e-a6ca-46a6-8545-aaf1e073bf04 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:29.928393 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4da261e-a6ca-46a6-8545-aaf1e073bf04 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:29.928648 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4da261e-a6ca-46a6-8545-aaf1e073bf04 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:29.930284 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 485/954] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:21:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:21:29.935018 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362ab95c-5a45-45e0-b22d-def3c7111803 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:29.935300 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362ab95c-5a45-45e0-b22d-def3c7111803 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:29.935575 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362ab95c-5a45-45e0-b22d-def3c7111803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:29.956255 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-362ab95c-5a45-45e0-b22d-def3c7111803 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:21:29.957559 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-362ab95c-5a45-45e0-b22d-def3c7111803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:21:29.996738 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:30.006319 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8fa670-027d-42eb-9b31-b0bc3da3090e None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:30.040693 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 486/955] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:21:29 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 07:21:30.047334 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d550b1a2-749f-450a-9742-c5604d4395d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:30.048117 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d550b1a2-749f-450a-9742-c5604d4395d3 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:30.048394 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d550b1a2-749f-450a-9742-c5604d4395d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:30.048670 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d550b1a2-749f-450a-9742-c5604d4395d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:30.067318 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d550b1a2-749f-450a-9742-c5604d4395d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:30.068881 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d550b1a2-749f-450a-9742-c5604d4395d3 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:30.086080 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-d550b1a2-749f-450a-9742-c5604d4395d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:21:30.086080 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:30.086080 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:30.086080 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:30.086080 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:30.086080 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:30.086080 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:30.086080 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:30.086080 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:30.086080 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:30.086080 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:30.086080 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:30.086080 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 07:21:30.086080 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 07:21:30.086080 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 07:21:30.086080 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 07:21:30.087059 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:30.087059 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:30.087059 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:30.087059 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:30.087059 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:30.087059 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:30.087059 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:30.087059 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:30.087059 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:30.087059 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:30.087059 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:30.087059 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:30.087059 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:30.087059 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:30.087059 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:30.087059 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:30.087059 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 07:21:30.087954 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 07:21:30.087954 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:21:30.087954 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:21:30.087954 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 470/956] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:21:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:21:30.093180 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362ab95c-5a45-45e0-b22d-def3c7111803 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:30.177448 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7707df-8b17-4e52-a893-ae6be96643cc None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:30.177835 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7707df-8b17-4e52-a893-ae6be96643cc None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:30.178041 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7707df-8b17-4e52-a893-ae6be96643cc None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:30.179838 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 471/957] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:21:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:21:30.184685 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6753f9-d7fa-47d8-a039-8473f57a6e3a None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:30.184974 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6753f9-d7fa-47d8-a039-8473f57a6e3a None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:30.185256 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6753f9-d7fa-47d8-a039-8473f57a6e3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:30.196331 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:30.206587 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362ab95c-5a45-45e0-b22d-def3c7111803 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:30.216115 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-2b6753f9-d7fa-47d8-a039-8473f57a6e3a None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:21:30.217694 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-2b6753f9-d7fa-47d8-a039-8473f57a6e3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:21:30.249706 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef6ddce-55c8-4d48-bc11-364d805bbc8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:30.251260 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef6ddce-55c8-4d48-bc11-364d805bbc8e None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:30.251857 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef6ddce-55c8-4d48-bc11-364d805bbc8e None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:30.252353 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef6ddce-55c8-4d48-bc11-364d805bbc8e None admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:30.253204 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef6ddce-55c8-4d48-bc11-364d805bbc8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:30.253749 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef6ddce-55c8-4d48-bc11-364d805bbc8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:30.255481 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef6ddce-55c8-4d48-bc11-364d805bbc8e None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:30.268486 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 487/958] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 15 07:21:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:30.277614 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef6ddce-55c8-4d48-bc11-364d805bbc8e None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:30.303346 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 472/959] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:21:30 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 07:21:30.314865 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:30.351526 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6753f9-d7fa-47d8-a039-8473f57a6e3a None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:30.397753 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef6ddce-55c8-4d48-bc11-364d805bbc8e None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:30.435030 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9bfbd2-cabe-40a2-bdea-88c689033db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:30.435663 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9bfbd2-cabe-40a2-bdea-88c689033db3 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:30.435950 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9bfbd2-cabe-40a2-bdea-88c689033db3 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:30.436337 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9bfbd2-cabe-40a2-bdea-88c689033db3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:30.445020 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9bfbd2-cabe-40a2-bdea-88c689033db3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:30.446500 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9bfbd2-cabe-40a2-bdea-88c689033db3 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:30.459402 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-ec9bfbd2-cabe-40a2-bdea-88c689033db3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 15 07:21:30.459402 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:30.459402 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:30.459402 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:30.459402 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:30.459402 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:30.459402 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:30.459402 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:30.459402 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:30.459402 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:30.459402 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:30.459402 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:30.459402 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 07:21:30.459402 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 07:21:30.459402 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 07:21:30.459402 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 07:21:30.460353 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:30.460353 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:30.460353 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 07:21:30.460353 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 07:21:30.460353 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 07:21:30.460353 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 07:21:30.460353 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:30.460353 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:30.460353 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:30.460353 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:30.460353 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:30.460353 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:30.460353 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:30.460353 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:30.460353 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:30.460353 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:30.460353 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:30.460353 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:30.461331 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:30.461331 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:30.461331 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 07:21:30.461331 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 07:21:30.461331 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 07:21:30.461331 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 07:21:30.461331 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 07:21:30.461331 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 07:21:30.461331 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 15 07:21:30.461331 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:21:30.461331 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 488/960] 10.222.0.42 () {56 vars in 1117 bytes} [Thu Sep 15 07:21:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:21:30.462004 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6753f9-d7fa-47d8-a039-8473f57a6e3a None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:30.467510 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9bfbd2-cabe-40a2-bdea-88c689033db3 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:30.498491 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154a296e-97e7-4747-9da6-a9b10e99999a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:30.499404 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154a296e-97e7-4747-9da6-a9b10e99999a None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:30.499887 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154a296e-97e7-4747-9da6-a9b10e99999a None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:30.500449 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154a296e-97e7-4747-9da6-a9b10e99999a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:30.501680 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154a296e-97e7-4747-9da6-a9b10e99999a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:30.503228 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154a296e-97e7-4747-9da6-a9b10e99999a None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:30.505257 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:30.515333 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-154a296e-97e7-4747-9da6-a9b10e99999a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 07:21:30.515333 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:30.515333 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 07:21:30.515333 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 07:21:30.515333 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 07:21:30.515333 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 07:21:30.515333 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 07:21:30.515333 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 07:21:30.515333 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 07:21:30.515333 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:21:30.515333 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 07:21:30.515333 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:21:30.515333 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:30.515333 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:30.515333 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:30.515333 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:30.515333 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:30.518899 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:30.518899 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:30.518899 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:30.518899 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:30.518899 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:30.518899 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:30.518899 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 07:21:30.518899 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 07:21:30.518899 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 07:21:30.518899 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 07:21:30.518899 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:30.518899 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:30.518899 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:30.518899 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:30.518899 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:30.518899 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:30.519974 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:30.519974 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:30.519974 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:30.519974 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:30.519974 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:30.519974 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:30.519974 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:30.519974 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:30.519974 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:30.519974 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:30.519974 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 07:21:30.519974 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 07:21:30.519974 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 07:21:30.519974 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:21:30.522124 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 473/961] 10.222.0.42 () {56 vars in 1126 bytes} [Thu Sep 15 07:21:30 2022] GET /identity/v3/services/object-store => generated 93 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:21:30.523100 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154a296e-97e7-4747-9da6-a9b10e99999a None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:30.572636 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:30.587335 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9bfbd2-cabe-40a2-bdea-88c689033db3 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:30.626507 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff080211-a842-4800-a3fc-f95dfdfc822c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:30.627039 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff080211-a842-4800-a3fc-f95dfdfc822c None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:30.627322 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff080211-a842-4800-a3fc-f95dfdfc822c None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:30.627596 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff080211-a842-4800-a3fc-f95dfdfc822c None admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:30.628326 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff080211-a842-4800-a3fc-f95dfdfc822c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:30.628634 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff080211-a842-4800-a3fc-f95dfdfc822c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:30.630118 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff080211-a842-4800-a3fc-f95dfdfc822c None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:30.633299 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154a296e-97e7-4747-9da6-a9b10e99999a None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:30.648579 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 489/962] 10.222.0.42 () {56 vars in 1149 bytes} [Thu Sep 15 07:21:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:30.649406 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff080211-a842-4800-a3fc-f95dfdfc822c None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:30.676256 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd46629-15f3-43ea-93be-203c8376f51e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:30.677112 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd46629-15f3-43ea-93be-203c8376f51e None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:30.677554 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd46629-15f3-43ea-93be-203c8376f51e None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:30.677868 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd46629-15f3-43ea-93be-203c8376f51e None admin] PATH_INFO: `/v3/services` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:30.678506 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd46629-15f3-43ea-93be-203c8376f51e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:30.678826 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd46629-15f3-43ea-93be-203c8376f51e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:30.680157 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd46629-15f3-43ea-93be-203c8376f51e None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:30.685224 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 474/963] 10.222.0.42 () {56 vars in 1122 bytes} [Thu Sep 15 07:21:30 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:30.690443 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd46629-15f3-43ea-93be-203c8376f51e None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:30.756076 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff080211-a842-4800-a3fc-f95dfdfc822c None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:30.757195 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:30.793300 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d91041c-a50e-4b03-950b-b562cf55933d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:30.793978 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d91041c-a50e-4b03-950b-b562cf55933d None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:30.794259 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d91041c-a50e-4b03-950b-b562cf55933d None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:30.794534 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d91041c-a50e-4b03-950b-b562cf55933d None admin] PATH_INFO: `/v3/services/compute` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:30.795108 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d91041c-a50e-4b03-950b-b562cf55933d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:30.796474 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d91041c-a50e-4b03-950b-b562cf55933d None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:30.801406 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd46629-15f3-43ea-93be-203c8376f51e None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:30.805710 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-2d91041c-a50e-4b03-950b-b562cf55933d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 15 07:21:30.805710 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:30.805710 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 07:21:30.805710 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 07:21:30.805710 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 07:21:30.805710 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 07:21:30.805710 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 07:21:30.805710 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 07:21:30.805710 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 15 07:21:30.805710 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:21:30.805710 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 07:21:30.805710 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:21:30.805710 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:30.805710 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:30.805710 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:30.805710 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:30.805710 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:30.806666 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:30.806666 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:30.806666 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:30.806666 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:30.806666 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:30.806666 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:30.806666 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 07:21:30.806666 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 07:21:30.806666 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 07:21:30.806666 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 07:21:30.806666 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:30.806666 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:30.806666 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:30.806666 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:30.806666 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:30.806666 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:30.807485 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:30.807485 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:30.807485 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:30.807485 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:30.807485 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:30.807485 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:30.807485 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:30.807485 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:30.807485 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:30.807485 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:30.807485 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 07:21:30.807485 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 07:21:30.807485 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 15 07:21:30.807485 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:21:30.807485 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 490/964] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 15 07:21:30 2022] GET /identity/v3/services/compute => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:21:30.808273 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d91041c-a50e-4b03-950b-b562cf55933d None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:30.822946 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:30.840753 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38108d45-1dd3-4bb0-85ad-391b0392c593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:30.841377 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38108d45-1dd3-4bb0-85ad-391b0392c593 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:30.844076 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38108d45-1dd3-4bb0-85ad-391b0392c593 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:30.844076 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38108d45-1dd3-4bb0-85ad-391b0392c593 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:30.916344 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d91041c-a50e-4b03-950b-b562cf55933d None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:30.918446 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38108d45-1dd3-4bb0-85ad-391b0392c593 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:30.920057 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38108d45-1dd3-4bb0-85ad-391b0392c593 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:30.952316 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d7f41d-6c9f-4c64-9160-4c2de52237fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:30.952932 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d7f41d-6c9f-4c64-9160-4c2de52237fa None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:30.953217 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d7f41d-6c9f-4c64-9160-4c2de52237fa None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:30.953749 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d7f41d-6c9f-4c64-9160-4c2de52237fa None admin] PATH_INFO: `/v3/services` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:30.954262 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d7f41d-6c9f-4c64-9160-4c2de52237fa None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:30.954573 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d7f41d-6c9f-4c64-9160-4c2de52237fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:30.955956 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d7f41d-6c9f-4c64-9160-4c2de52237fa None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:30.966543 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 491/965] 10.222.0.42 () {56 vars in 1122 bytes} [Thu Sep 15 07:21:30 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:30.966543 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d7f41d-6c9f-4c64-9160-4c2de52237fa None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:30.990914 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:31.034785 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 475/966] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 15 07:21:30 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 07:21:31.035623 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38108d45-1dd3-4bb0-85ad-391b0392c593 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:31.050223 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:31.075840 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d7f41d-6c9f-4c64-9160-4c2de52237fa None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:31.113978 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0069ec8-4597-4b5d-ace7-f912d93a73cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:31.114588 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0069ec8-4597-4b5d-ace7-f912d93a73cb None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:31.114869 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0069ec8-4597-4b5d-ace7-f912d93a73cb None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:31.115309 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0069ec8-4597-4b5d-ace7-f912d93a73cb None admin] PATH_INFO: `/v3/services` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:31.115946 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0069ec8-4597-4b5d-ace7-f912d93a73cb None admin] RBAC: Adding query filter params (name=compute) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:31.116252 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0069ec8-4597-4b5d-ace7-f912d93a73cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:31.117640 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0069ec8-4597-4b5d-ace7-f912d93a73cb None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:31.127517 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 492/967] 10.222.0.42 () {56 vars in 1112 bytes} [Thu Sep 15 07:21:30 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:31.127517 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0069ec8-4597-4b5d-ace7-f912d93a73cb None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:31.152545 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38108d45-1dd3-4bb0-85ad-391b0392c593 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:31.191056 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e924ec-7baa-4eb4-af62-7834ff0c7e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:31.191855 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e924ec-7baa-4eb4-af62-7834ff0c7e09 None admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:31.192138 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e924ec-7baa-4eb4-af62-7834ff0c7e09 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:31.192403 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e924ec-7baa-4eb4-af62-7834ff0c7e09 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:31.193099 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e924ec-7baa-4eb4-af62-7834ff0c7e09 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:31.194486 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e924ec-7baa-4eb4-af62-7834ff0c7e09 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:31.233489 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0069ec8-4597-4b5d-ace7-f912d93a73cb None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:31.268236 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619b6da9-dcb0-4ec5-b278-eb0a8cf01045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:31.268868 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b6da9-dcb0-4ec5-b278-eb0a8cf01045 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:31.269151 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b6da9-dcb0-4ec5-b278-eb0a8cf01045 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:31.269425 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b6da9-dcb0-4ec5-b278-eb0a8cf01045 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:31.278816 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:31.281464 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619b6da9-dcb0-4ec5-b278-eb0a8cf01045 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:31.283998 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619b6da9-dcb0-4ec5-b278-eb0a8cf01045 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:31.293911 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 476/968] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:21:31 2022] POST /identity/v3/endpoints => generated 349 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:21:31.293911 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e924ec-7baa-4eb4-af62-7834ff0c7e09 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:31.299758 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-619b6da9-dcb0-4ec5-b278-eb0a8cf01045 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:21:31.299758 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:31.299758 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:31.299758 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:31.299758 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:31.299758 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:31.299758 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:31.299758 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:31.299758 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:31.299758 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:31.299758 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:31.299758 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:31.299758 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 07:21:31.299758 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 07:21:31.299758 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 07:21:31.299758 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 07:21:31.300775 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:31.300775 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:31.300775 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:31.300775 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:31.300775 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:31.300775 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:31.300775 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:31.300775 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:31.300775 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:31.300775 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:31.300775 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:31.300775 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:31.300775 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:31.300775 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:31.300775 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:31.300775 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:31.300775 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 07:21:31.301727 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 07:21:31.301727 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:21:31.301727 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:21:31.301727 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 493/969] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:21:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:21:31.308417 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619b6da9-dcb0-4ec5-b278-eb0a8cf01045 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:31.317948 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:31.439065 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e924ec-7baa-4eb4-af62-7834ff0c7e09 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:31.465336 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619b6da9-dcb0-4ec5-b278-eb0a8cf01045 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:31.488594 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c75e3df-f31d-43ad-a3cb-9bcbefa57c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:31.489214 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c75e3df-f31d-43ad-a3cb-9bcbefa57c66 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:31.490005 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c75e3df-f31d-43ad-a3cb-9bcbefa57c66 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:31.490283 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c75e3df-f31d-43ad-a3cb-9bcbefa57c66 None admin] PATH_INFO: `/v3/services` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:31.491076 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c75e3df-f31d-43ad-a3cb-9bcbefa57c66 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:31.491664 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c75e3df-f31d-43ad-a3cb-9bcbefa57c66 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:31.493509 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c75e3df-f31d-43ad-a3cb-9bcbefa57c66 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:31.499174 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 477/970] 10.222.0.42 () {56 vars in 1112 bytes} [Thu Sep 15 07:21:31 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:31.504931 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dad31fe-7d35-4005-87a1-7d014292319f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:31.505709 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dad31fe-7d35-4005-87a1-7d014292319f None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:31.506077 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dad31fe-7d35-4005-87a1-7d014292319f None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:31.506421 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dad31fe-7d35-4005-87a1-7d014292319f None admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:31.507157 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dad31fe-7d35-4005-87a1-7d014292319f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:31.507530 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dad31fe-7d35-4005-87a1-7d014292319f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:31.509002 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dad31fe-7d35-4005-87a1-7d014292319f None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:31.523150 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 494/971] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 15 07:21:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:31.529630 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c75e3df-f31d-43ad-a3cb-9bcbefa57c66 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:31.531127 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c29754-297c-4fa0-bc79-c78c8e78543c None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:31.531415 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c29754-297c-4fa0-bc79-c78c8e78543c None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:31.531693 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c29754-297c-4fa0-bc79-c78c8e78543c None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:31.535466 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 495/972] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:21:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:21:31.540014 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d57b402-4708-4da8-b159-ed23efc46907 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:31.540298 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d57b402-4708-4da8-b159-ed23efc46907 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:31.540569 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d57b402-4708-4da8-b159-ed23efc46907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:31.563243 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-6d57b402-4708-4da8-b159-ed23efc46907 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:21:31.565408 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-6d57b402-4708-4da8-b159-ed23efc46907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:21:31.621528 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:31.640340 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:31.649723 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 496/973] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:21:31 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 07:21:31.667113 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c75e3df-f31d-43ad-a3cb-9bcbefa57c66 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:31.706017 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7f0cbe-e7e7-4f73-a5fa-0f8ee17a7f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:31.706659 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f0cbe-e7e7-4f73-a5fa-0f8ee17a7f58 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:31.706938 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f0cbe-e7e7-4f73-a5fa-0f8ee17a7f58 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:31.707210 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f0cbe-e7e7-4f73-a5fa-0f8ee17a7f58 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:31.714961 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7f0cbe-e7e7-4f73-a5fa-0f8ee17a7f58 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:31.716511 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7f0cbe-e7e7-4f73-a5fa-0f8ee17a7f58 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:31.728409 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-7e7f0cbe-e7e7-4f73-a5fa-0f8ee17a7f58 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 07:21:31.728409 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:31.728409 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:31.728409 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:31.728409 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:31.728409 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:31.728409 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:31.728409 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:31.728409 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:31.728409 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:31.728409 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:31.728409 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:31.728409 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 07:21:31.728409 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 07:21:31.728409 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 07:21:31.729359 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 07:21:31.729359 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:31.729359 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:31.729359 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:31.729359 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:31.729359 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:31.729359 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:31.729359 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:31.729359 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:31.729359 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:31.729359 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:31.729359 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:31.729359 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:31.729359 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:31.729359 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:31.729359 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:31.729359 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:31.730285 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 07:21:31.730285 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 07:21:31.730285 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 07:21:31.730285 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 07:21:31.730285 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 07:21:31.730285 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 07:21:31.730285 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 07:21:31.730285 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:21:31.730285 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 478/974] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 15 07:21:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:21:31.735534 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d57b402-4708-4da8-b159-ed23efc46907 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:31.852442 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d57b402-4708-4da8-b159-ed23efc46907 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:31.889798 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9188304a-561f-43e6-96d0-c7fbe34a890a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:31.890498 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9188304a-561f-43e6-96d0-c7fbe34a890a None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:31.890700 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9188304a-561f-43e6-96d0-c7fbe34a890a None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:31.890978 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9188304a-561f-43e6-96d0-c7fbe34a890a None admin] PATH_INFO: `/v3/projects` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:31.892261 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9188304a-561f-43e6-96d0-c7fbe34a890a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:31.892575 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9188304a-561f-43e6-96d0-c7fbe34a890a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:31.894048 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9188304a-561f-43e6-96d0-c7fbe34a890a None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:31.906994 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 497/975] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 15 07:21:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:31.913752 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7f0cbe-e7e7-4f73-a5fa-0f8ee17a7f58 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:31.926728 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9188304a-561f-43e6-96d0-c7fbe34a890a None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:31.990497 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:32.043851 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7f0cbe-e7e7-4f73-a5fa-0f8ee17a7f58 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:32.072334 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9188304a-561f-43e6-96d0-c7fbe34a890a None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:32.094660 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c7e354-3c29-499f-a4b8-257970c5a4fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:32.095741 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c7e354-3c29-499f-a4b8-257970c5a4fa None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:32.096224 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c7e354-3c29-499f-a4b8-257970c5a4fa None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:32.096739 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c7e354-3c29-499f-a4b8-257970c5a4fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:32.097691 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c7e354-3c29-499f-a4b8-257970c5a4fa None admin] RBAC: Adding query filter params (role.id=0888721641c54c3684692df3615ea1b0, scope.project.id=4e9987720930473da840579654311bc2, user.id=f9c68c4bd9074d8d9b893755e28c0807) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:32.098219 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c7e354-3c29-499f-a4b8-257970c5a4fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:32.100805 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c7e354-3c29-499f-a4b8-257970c5a4fa None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:32.107386 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903412bb-51fa-4353-b2e8-b33c16886110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:32.107995 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903412bb-51fa-4353-b2e8-b33c16886110 None admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:32.108406 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903412bb-51fa-4353-b2e8-b33c16886110 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:32.108604 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903412bb-51fa-4353-b2e8-b33c16886110 None admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:32.108968 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 479/976] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 15 07:21:31 2022] GET /identity/v3/role_assignments?user.id=f9c68c4bd9074d8d9b893755e28c0807&scope.project.id=4e9987720930473da840579654311bc2&role.id=0888721641c54c3684692df3615ea1b0 => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:32.109175 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903412bb-51fa-4353-b2e8-b33c16886110 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:32.111175 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903412bb-51fa-4353-b2e8-b33c16886110 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:32.120880 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: WARNING keystone.server.flask.common [None req-903412bb-51fa-4353-b2e8-b33c16886110 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 15 07:21:32.163630 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.sql.core [None req-903412bb-51fa-4353-b2e8-b33c16886110 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 15 07:21:32.163630 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 15 07:21:32.163630 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [parameters: {'user_id': '6c849d7fe0b14be98a8bd64438a59724', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 15 07:21:32.163630 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81598) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 15 07:21:32.182587 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-903412bb-51fa-4353-b2e8-b33c16886110 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 15 07:21:32.182587 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:32.182587 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 15 07:21:32.182587 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 15 07:21:32.182587 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 15 07:21:32.182587 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 15 07:21:32.182587 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 15 07:21:32.182587 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application result = self._query(query) Sep 15 07:21:32.182587 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 15 07:21:32.182587 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application conn.query(q) Sep 15 07:21:32.182587 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 15 07:21:32.182587 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 15 07:21:32.182587 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 15 07:21:32.182587 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application result.read() Sep 15 07:21:32.182587 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 15 07:21:32.183555 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 15 07:21:32.183555 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 15 07:21:32.183555 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 15 07:21:32.183555 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 15 07:21:32.183555 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 15 07:21:32.183555 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 15 07:21:32.183555 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 15 07:21:32.183555 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 15 07:21:32.183555 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:21:32.183555 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 15 07:21:32.183555 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:21:32.183555 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:32.183555 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 15 07:21:32.183555 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 15 07:21:32.183555 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 15 07:21:32.183555 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 15 07:21:32.183555 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 15 07:21:32.183555 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application next(self.gen) Sep 15 07:21:32.184947 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 15 07:21:32.184947 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application yield resource Sep 15 07:21:32.184947 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 15 07:21:32.184947 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application next(self.gen) Sep 15 07:21:32.184947 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 15 07:21:32.184947 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application self.session.rollback() Sep 15 07:21:32.184947 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 15 07:21:32.184947 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application self.force_reraise() Sep 15 07:21:32.184947 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 15 07:21:32.184947 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise self.value Sep 15 07:21:32.184947 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 15 07:21:32.184947 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 15 07:21:32.184947 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 15 07:21:32.184947 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application session.commit() Sep 15 07:21:32.184947 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 15 07:21:32.184947 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 15 07:21:32.184947 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 15 07:21:32.185852 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application self._prepare_impl() Sep 15 07:21:32.185852 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 15 07:21:32.185852 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application self.session.flush() Sep 15 07:21:32.185852 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 15 07:21:32.185852 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application self._flush(objects) Sep 15 07:21:32.185852 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 15 07:21:32.185852 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 15 07:21:32.185852 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 15 07:21:32.185852 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application compat.raise_( Sep 15 07:21:32.185852 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 15 07:21:32.185852 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception Sep 15 07:21:32.185852 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 15 07:21:32.185852 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application flush_context.execute() Sep 15 07:21:32.185852 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 15 07:21:32.185852 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rec.execute(self) Sep 15 07:21:32.185852 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 15 07:21:32.185852 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 15 07:21:32.186934 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 15 07:21:32.186934 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 15 07:21:32.186934 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 15 07:21:32.186934 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 15 07:21:32.186934 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 15 07:21:32.186934 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 15 07:21:32.186934 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 15 07:21:32.186934 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 15 07:21:32.186934 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 15 07:21:32.186934 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 15 07:21:32.186934 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 15 07:21:32.186934 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 15 07:21:32.186934 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 15 07:21:32.186934 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 15 07:21:32.186934 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 15 07:21:32.186934 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception Sep 15 07:21:32.188049 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 15 07:21:32.188049 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 15 07:21:32.188049 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 15 07:21:32.188049 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 15 07:21:32.188049 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 15 07:21:32.188049 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application result = self._query(query) Sep 15 07:21:32.188049 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 15 07:21:32.188049 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application conn.query(q) Sep 15 07:21:32.188049 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 15 07:21:32.188049 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 15 07:21:32.188049 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 15 07:21:32.188049 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application result.read() Sep 15 07:21:32.188049 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 15 07:21:32.188049 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 15 07:21:32.188049 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 15 07:21:32.188049 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 15 07:21:32.190765 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 15 07:21:32.190765 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 15 07:21:32.190765 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 15 07:21:32.190765 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 15 07:21:32.190765 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 15 07:21:32.190765 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 15 07:21:32.190765 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [parameters: {'user_id': '6c849d7fe0b14be98a8bd64438a59724', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 15 07:21:32.190765 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 15 07:21:32.190765 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:21:32.190765 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 07:21:32.190765 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:21:32.190765 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:32.190765 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:32.190765 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:32.190765 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:32.192628 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:32.192628 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:32.192628 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:32.192628 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:32.192628 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:32.192628 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:32.192628 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:32.192628 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 15 07:21:32.192628 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 15 07:21:32.192628 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:32.192628 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:32.192628 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 07:21:32.192628 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 07:21:32.192628 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 07:21:32.192628 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 07:21:32.192628 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 15 07:21:32.192628 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 15 07:21:32.194808 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 15 07:21:32.194808 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 15 07:21:32.194808 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 15 07:21:32.194808 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 15 07:21:32.194808 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 15 07:21:32.194808 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:21:32.194808 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 498/977] 10.222.0.42 () {60 vars in 1131 bytes} [Thu Sep 15 07:21:31 2022] POST /identity/v3/users => generated 160 bytes in 258 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 15 07:21:32.202508 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c7e354-3c29-499f-a4b8-257970c5a4fa None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:32.205614 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:32.316034 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c7e354-3c29-499f-a4b8-257970c5a4fa None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:32.355792 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01a0153-9065-4d4e-ab25-60df01034ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:32.356777 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01a0153-9065-4d4e-ab25-60df01034ec4 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:32.357256 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01a0153-9065-4d4e-ab25-60df01034ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:32.357753 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01a0153-9065-4d4e-ab25-60df01034ec4 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:32.366932 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01a0153-9065-4d4e-ab25-60df01034ec4 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:32.368676 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01a0153-9065-4d4e-ab25-60df01034ec4 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:32.380857 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-c01a0153-9065-4d4e-ab25-60df01034ec4 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 15 07:21:32.380857 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:32.380857 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:32.380857 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:32.380857 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:32.380857 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:32.380857 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:32.380857 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:32.380857 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:32.380857 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:32.380857 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:32.380857 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:32.380857 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 07:21:32.380857 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 07:21:32.380857 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 07:21:32.380857 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 07:21:32.381855 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:32.381855 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:32.381855 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 07:21:32.381855 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 07:21:32.381855 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 07:21:32.381855 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 07:21:32.381855 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:32.381855 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:32.381855 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:32.381855 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:32.381855 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:32.381855 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:32.381855 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:32.381855 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:32.381855 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:32.381855 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:32.381855 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:32.381855 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:32.382811 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:32.382811 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:32.382811 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 07:21:32.382811 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 07:21:32.382811 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 07:21:32.382811 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 07:21:32.382811 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 07:21:32.382811 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 07:21:32.382811 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 15 07:21:32.382811 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:21:32.389771 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 480/978] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 15 07:21:32 2022] GET /identity/v3/users/cinder => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:21:32.391668 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903412bb-51fa-4353-b2e8-b33c16886110 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:32.498637 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903412bb-51fa-4353-b2e8-b33c16886110 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:32.500791 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:32.542583 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1df7c70-5b47-41e5-b8e3-17a2b6dfbe1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:32.543201 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1df7c70-5b47-41e5-b8e3-17a2b6dfbe1e None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:32.543480 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1df7c70-5b47-41e5-b8e3-17a2b6dfbe1e None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:32.543751 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1df7c70-5b47-41e5-b8e3-17a2b6dfbe1e None admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:32.544440 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1df7c70-5b47-41e5-b8e3-17a2b6dfbe1e None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:32.544743 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1df7c70-5b47-41e5-b8e3-17a2b6dfbe1e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:32.546434 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1df7c70-5b47-41e5-b8e3-17a2b6dfbe1e None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:32.588111 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 499/979] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Sep 15 07:21:32 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:33.615539 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fca218-8057-4b5e-87ff-751ed07b6463 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:33.616058 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fca218-8057-4b5e-87ff-751ed07b6463 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:33.616169 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fca218-8057-4b5e-87ff-751ed07b6463 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:33.617463 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 481/980] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:21:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:21:33.622694 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e69e0f-5308-463d-8fca-5f34593240f2 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:33.623090 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e69e0f-5308-463d-8fca-5f34593240f2 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:33.623261 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e69e0f-5308-463d-8fca-5f34593240f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:33.642545 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-d7e69e0f-5308-463d-8fca-5f34593240f2 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:21:33.644152 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-d7e69e0f-5308-463d-8fca-5f34593240f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:21:33.717595 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 500/981] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:21:33 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 07:21:33.776901 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d2d682-0b8a-4aaa-880d-62172898261a None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:33.777206 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d2d682-0b8a-4aaa-880d-62172898261a None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:33.777465 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d2d682-0b8a-4aaa-880d-62172898261a None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:33.779046 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 482/982] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:21:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:21:33.784784 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b651d0-e324-4250-a4bb-0f11aeccc36b None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:33.785072 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b651d0-e324-4250-a4bb-0f11aeccc36b None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:33.785350 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b651d0-e324-4250-a4bb-0f11aeccc36b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:33.804072 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-42b651d0-e324-4250-a4bb-0f11aeccc36b None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:21:33.805223 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-42b651d0-e324-4250-a4bb-0f11aeccc36b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:21:33.871613 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 501/983] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:21:33 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 07:21:33.990200 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d2d682-0b8a-4aaa-880d-62172898261a None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:34.056607 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:34.099798 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d2d682-0b8a-4aaa-880d-62172898261a None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:34.135052 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ae145e-ddc3-41fe-b5b6-63a605cc4fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:34.135608 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae145e-ddc3-41fe-b5b6-63a605cc4fd5 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:34.135892 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae145e-ddc3-41fe-b5b6-63a605cc4fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:34.136169 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae145e-ddc3-41fe-b5b6-63a605cc4fd5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:34.136754 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ae145e-ddc3-41fe-b5b6-63a605cc4fd5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:34.138167 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ae145e-ddc3-41fe-b5b6-63a605cc4fd5 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:34.141214 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b651d0-e324-4250-a4bb-0f11aeccc36b None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:34.147270 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-e5ae145e-ddc3-41fe-b5b6-63a605cc4fd5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 07:21:34.147270 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:34.147270 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 07:21:34.147270 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 07:21:34.147270 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 07:21:34.147270 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 07:21:34.147270 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 07:21:34.147270 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 07:21:34.147270 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 07:21:34.147270 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:21:34.147270 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 07:21:34.147270 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:21:34.147270 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:34.147270 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:34.147270 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:34.147270 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:34.147270 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:34.148241 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:34.148241 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:34.148241 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:34.148241 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:34.148241 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:34.148241 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:34.148241 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 07:21:34.148241 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 07:21:34.148241 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 07:21:34.148241 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 07:21:34.148241 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:34.148241 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:34.148241 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:34.148241 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:34.148241 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:34.148241 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:34.149078 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:34.149078 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:34.149078 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:34.149078 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:34.149078 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:34.149078 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:34.149078 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:34.149078 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:34.149078 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:34.149078 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:34.149078 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 07:21:34.149078 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 07:21:34.149078 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 07:21:34.149078 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:21:34.149836 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 483/984] 10.222.0.42 () {56 vars in 1126 bytes} [Thu Sep 15 07:21:33 2022] GET /identity/v3/services/object-store => generated 93 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:21:34.154467 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ae145e-ddc3-41fe-b5b6-63a605cc4fd5 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:34.164107 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:34.250144 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b651d0-e324-4250-a4bb-0f11aeccc36b None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:34.260010 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ae145e-ddc3-41fe-b5b6-63a605cc4fd5 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:34.287097 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2b8cd7-7c73-4364-a94d-45273fa0d49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:34.287723 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2b8cd7-7c73-4364-a94d-45273fa0d49c None admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:34.288165 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2b8cd7-7c73-4364-a94d-45273fa0d49c None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:34.288800 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:34.289090 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2b8cd7-7c73-4364-a94d-45273fa0d49c None admin] PATH_INFO: `/v3/services` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:34.290747 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2b8cd7-7c73-4364-a94d-45273fa0d49c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:34.292367 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2b8cd7-7c73-4364-a94d-45273fa0d49c None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:34.298900 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16baa7e-4231-4689-b862-e754705b6c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:34.299546 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16baa7e-4231-4689-b862-e754705b6c0a None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:34.299825 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16baa7e-4231-4689-b862-e754705b6c0a None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:34.300113 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16baa7e-4231-4689-b862-e754705b6c0a None admin] PATH_INFO: `/v3/services` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:34.300765 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16baa7e-4231-4689-b862-e754705b6c0a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:34.304294 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16baa7e-4231-4689-b862-e754705b6c0a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:34.304294 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16baa7e-4231-4689-b862-e754705b6c0a None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:34.307991 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 484/985] 10.222.0.42 () {56 vars in 1122 bytes} [Thu Sep 15 07:21:34 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:34.314793 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16baa7e-4231-4689-b862-e754705b6c0a None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:34.382543 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 502/986] 10.222.0.42 () {60 vars in 1141 bytes} [Thu Sep 15 07:21:34 2022] POST /identity/v3/services => generated 242 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 07:21:34.415240 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31dc4f7-fd6b-478d-9f57-7ac24175a9bd None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:34.415240 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31dc4f7-fd6b-478d-9f57-7ac24175a9bd None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:34.415240 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31dc4f7-fd6b-478d-9f57-7ac24175a9bd None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:34.415240 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 503/987] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:21:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:21:34.418667 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f440fe1-5a33-4384-b5c3-fd5070306f5d None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:34.418977 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f440fe1-5a33-4384-b5c3-fd5070306f5d None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:34.419375 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f440fe1-5a33-4384-b5c3-fd5070306f5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:34.427474 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16baa7e-4231-4689-b862-e754705b6c0a None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:34.441608 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-1f440fe1-5a33-4384-b5c3-fd5070306f5d None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:21:34.443052 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-1f440fe1-5a33-4384-b5c3-fd5070306f5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:21:34.476341 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80f833c-85ca-4d05-9377-9c10f6388cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:34.477074 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80f833c-85ca-4d05-9377-9c10f6388cbf None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:34.477364 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80f833c-85ca-4d05-9377-9c10f6388cbf None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:34.477688 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80f833c-85ca-4d05-9377-9c10f6388cbf None admin] PATH_INFO: `/v3/services` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:34.478343 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80f833c-85ca-4d05-9377-9c10f6388cbf None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:34.478676 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80f833c-85ca-4d05-9377-9c10f6388cbf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:34.480153 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80f833c-85ca-4d05-9377-9c10f6388cbf None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:34.489773 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 485/988] 10.222.0.42 () {56 vars in 1122 bytes} [Thu Sep 15 07:21:34 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:21:34.493389 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80f833c-85ca-4d05-9377-9c10f6388cbf None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:34.521608 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 504/989] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:21:34 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 07:21:34.553928 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:34.606086 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80f833c-85ca-4d05-9377-9c10f6388cbf None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:34.640540 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72cc62c-e57d-4b70-beb6-6bc42eeffeb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:34.641159 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72cc62c-e57d-4b70-beb6-6bc42eeffeb3 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:34.641418 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72cc62c-e57d-4b70-beb6-6bc42eeffeb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:34.641795 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72cc62c-e57d-4b70-beb6-6bc42eeffeb3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:34.642500 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72cc62c-e57d-4b70-beb6-6bc42eeffeb3 None admin] RBAC: Adding query filter params (interface=admin, service_id=1510b5edd14248b4ac75514172d6ea2b, region_id=RegionOne) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:34.642807 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72cc62c-e57d-4b70-beb6-6bc42eeffeb3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:34.644152 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72cc62c-e57d-4b70-beb6-6bc42eeffeb3 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:34.653650 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 486/990] 10.222.0.42 () {56 vars in 1249 bytes} [Thu Sep 15 07:21:34 2022] GET /identity/v3/endpoints?service_id=1510b5edd14248b4ac75514172d6ea2b&interface=admin®ion_id=RegionOne => generated 193 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:34.655446 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f440fe1-5a33-4384-b5c3-fd5070306f5d None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:34.668092 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:34.758750 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f440fe1-5a33-4384-b5c3-fd5070306f5d None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:34.792415 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72cc62c-e57d-4b70-beb6-6bc42eeffeb3 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:34.794793 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d432807-ad60-48ac-8897-10f72a4cc4bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:34.795635 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d432807-ad60-48ac-8897-10f72a4cc4bc None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:34.795921 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d432807-ad60-48ac-8897-10f72a4cc4bc None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:34.797100 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d432807-ad60-48ac-8897-10f72a4cc4bc None admin] PATH_INFO: `/v3/services` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:34.797680 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d432807-ad60-48ac-8897-10f72a4cc4bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:34.799978 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d432807-ad60-48ac-8897-10f72a4cc4bc None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:34.813820 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 505/991] 10.222.0.42 () {56 vars in 1087 bytes} [Thu Sep 15 07:21:34 2022] GET /identity/v3/services => generated 1727 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:21:34.911245 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72cc62c-e57d-4b70-beb6-6bc42eeffeb3 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:34.927229 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:34.956169 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465dceff-1ee8-42cc-a1f4-d2b1ab49c282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:34.956837 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465dceff-1ee8-42cc-a1f4-d2b1ab49c282 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:34.957186 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465dceff-1ee8-42cc-a1f4-d2b1ab49c282 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:34.957466 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465dceff-1ee8-42cc-a1f4-d2b1ab49c282 None admin] PATH_INFO: `/v3/services/image` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:34.958113 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465dceff-1ee8-42cc-a1f4-d2b1ab49c282 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:34.959503 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465dceff-1ee8-42cc-a1f4-d2b1ab49c282 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:34.971048 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-465dceff-1ee8-42cc-a1f4-d2b1ab49c282 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 15 07:21:34.971048 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:34.971048 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 07:21:34.971048 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 07:21:34.971048 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 07:21:34.971048 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 07:21:34.971048 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 07:21:34.971048 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 07:21:34.971048 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 15 07:21:34.971048 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:21:34.971048 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 07:21:34.971048 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:21:34.971048 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:34.971048 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:34.971048 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:34.971048 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:34.971048 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:34.972131 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:34.972131 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:34.972131 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:34.972131 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:34.972131 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:34.972131 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:34.972131 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 07:21:34.972131 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 07:21:34.972131 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 07:21:34.972131 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 07:21:34.972131 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:34.972131 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:34.972131 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:34.972131 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:34.972131 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:34.972131 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:34.972978 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:34.972978 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:34.972978 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:34.972978 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:34.972978 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:34.972978 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:34.972978 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:34.972978 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:34.972978 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:34.972978 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:34.972978 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 07:21:34.972978 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 07:21:34.972978 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 15 07:21:34.972978 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:21:34.977030 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 487/992] 10.222.0.42 () {56 vars in 1105 bytes} [Thu Sep 15 07:21:34 2022] GET /identity/v3/services/image => generated 86 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 07:21:34.978236 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d432807-ad60-48ac-8897-10f72a4cc4bc None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:35.062198 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:35.088357 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d432807-ad60-48ac-8897-10f72a4cc4bc None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:35.125141 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b66821-4c4e-4ae2-853c-12d9e9804b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:35.125786 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b66821-4c4e-4ae2-853c-12d9e9804b31 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:35.126096 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b66821-4c4e-4ae2-853c-12d9e9804b31 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:35.126942 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b66821-4c4e-4ae2-853c-12d9e9804b31 None admin] PATH_INFO: `/v3/services` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:35.127582 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b66821-4c4e-4ae2-853c-12d9e9804b31 None admin] RBAC: Adding query filter params (name=image) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:35.127889 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b66821-4c4e-4ae2-853c-12d9e9804b31 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:35.129311 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b66821-4c4e-4ae2-853c-12d9e9804b31 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:35.138821 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 506/993] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:21:34 2022] GET /identity/v3/services?name=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:35.139791 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465dceff-1ee8-42cc-a1f4-d2b1ab49c282 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:35.252831 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465dceff-1ee8-42cc-a1f4-d2b1ab49c282 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:35.287837 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3be69b-6c02-408c-96d6-5615348b7cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:35.288445 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3be69b-6c02-408c-96d6-5615348b7cb1 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:35.288728 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3be69b-6c02-408c-96d6-5615348b7cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:35.289000 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3be69b-6c02-408c-96d6-5615348b7cb1 None admin] PATH_INFO: `/v3/services` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:35.289657 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3be69b-6c02-408c-96d6-5615348b7cb1 None admin] RBAC: Adding query filter params (type=image) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:35.289964 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3be69b-6c02-408c-96d6-5615348b7cb1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:35.291362 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3be69b-6c02-408c-96d6-5615348b7cb1 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:35.300717 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 488/994] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:21:35 2022] GET /identity/v3/services?type=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:21:36.624887 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0763af5a-64d3-429a-a08b-a98499a0f58a None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:36.625440 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0763af5a-64d3-429a-a08b-a98499a0f58a None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:36.625440 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0763af5a-64d3-429a-a08b-a98499a0f58a None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:36.627035 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 507/995] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:21:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:21:36.632087 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8a5491-4f81-4674-977e-1b7fe6269cb7 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:36.632470 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8a5491-4f81-4674-977e-1b7fe6269cb7 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:36.632645 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8a5491-4f81-4674-977e-1b7fe6269cb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:36.651465 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-ac8a5491-4f81-4674-977e-1b7fe6269cb7 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:21:36.652838 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-ac8a5491-4f81-4674-977e-1b7fe6269cb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:21:36.690475 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:36.721336 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 489/996] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:21:36 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 07:21:36.992866 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0763af5a-64d3-429a-a08b-a98499a0f58a None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:37.000781 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a06bd84-89ea-4b13-a341-c26ea671ea12 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:37.001717 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a06bd84-89ea-4b13-a341-c26ea671ea12 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:37.002329 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a06bd84-89ea-4b13-a341-c26ea671ea12 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:37.003831 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 490/997] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:21:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 07:21:37.008829 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc3d5d3-7d4c-4cc5-9f2d-c715b2d7aaf4 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:37.009122 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc3d5d3-7d4c-4cc5-9f2d-c715b2d7aaf4 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:37.009404 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc3d5d3-7d4c-4cc5-9f2d-c715b2d7aaf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:37.028390 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-5bc3d5d3-7d4c-4cc5-9f2d-c715b2d7aaf4 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:21:37.029560 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-5bc3d5d3-7d4c-4cc5-9f2d-c715b2d7aaf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:21:37.101762 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 491/998] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:21:37 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 07:21:37.105511 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0763af5a-64d3-429a-a08b-a98499a0f58a None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:37.144150 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95abe53e-c8cf-490e-83d0-b90cc8366272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:37.144784 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95abe53e-c8cf-490e-83d0-b90cc8366272 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:37.145063 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95abe53e-c8cf-490e-83d0-b90cc8366272 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:37.145334 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95abe53e-c8cf-490e-83d0-b90cc8366272 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:37.145937 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95abe53e-c8cf-490e-83d0-b90cc8366272 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:37.147300 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95abe53e-c8cf-490e-83d0-b90cc8366272 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:37.156134 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-95abe53e-c8cf-490e-83d0-b90cc8366272 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 15 07:21:37.156134 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:37.156134 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 07:21:37.156134 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 07:21:37.156134 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 07:21:37.156134 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 07:21:37.156134 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 07:21:37.156134 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 07:21:37.156134 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 15 07:21:37.156134 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:21:37.156134 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 07:21:37.156134 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:21:37.156134 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:37.156134 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:37.156134 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:37.156134 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:37.156134 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:37.157156 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:37.157156 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:37.157156 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:37.157156 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:37.157156 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:37.157156 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:37.157156 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 07:21:37.157156 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 07:21:37.157156 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 07:21:37.157156 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 07:21:37.157156 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:37.157156 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:37.157156 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:37.157156 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:37.157156 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:37.157156 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:37.158612 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:37.158612 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:37.158612 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:37.158612 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:37.158612 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:37.158612 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:37.158612 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:37.158612 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:37.158612 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:37.158612 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:37.158612 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 07:21:37.158612 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 07:21:37.158612 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 15 07:21:37.158612 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:21:37.161936 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 508/999] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 15 07:21:36 2022] GET /identity/v3/services/compute => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:21:37.163951 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc3d5d3-7d4c-4cc5-9f2d-c715b2d7aaf4 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:37.270431 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc3d5d3-7d4c-4cc5-9f2d-c715b2d7aaf4 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:37.272055 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:37.304349 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13647f9-8bdf-4ebb-b709-e81e8dd2540e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:37.304961 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13647f9-8bdf-4ebb-b709-e81e8dd2540e None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:37.305664 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13647f9-8bdf-4ebb-b709-e81e8dd2540e None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:37.306310 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13647f9-8bdf-4ebb-b709-e81e8dd2540e None admin] PATH_INFO: `/v3/services` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:37.307211 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13647f9-8bdf-4ebb-b709-e81e8dd2540e None admin] RBAC: Adding query filter params (name=compute) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:37.308054 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13647f9-8bdf-4ebb-b709-e81e8dd2540e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:37.309468 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13647f9-8bdf-4ebb-b709-e81e8dd2540e None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:37.315703 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 492/1000] 10.222.0.42 () {56 vars in 1112 bytes} [Thu Sep 15 07:21:37 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:21:37.321740 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95abe53e-c8cf-490e-83d0-b90cc8366272 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:37.325025 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:37.374665 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13647f9-8bdf-4ebb-b709-e81e8dd2540e None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:37.435433 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95abe53e-c8cf-490e-83d0-b90cc8366272 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:37.469393 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8b208c-13d3-441b-a199-0387bad7b361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:37.470639 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8b208c-13d3-441b-a199-0387bad7b361 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:37.470924 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8b208c-13d3-441b-a199-0387bad7b361 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:37.471203 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8b208c-13d3-441b-a199-0387bad7b361 None admin] PATH_INFO: `/v3/services` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:37.471827 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8b208c-13d3-441b-a199-0387bad7b361 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:37.472133 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8b208c-13d3-441b-a199-0387bad7b361 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:37.473505 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8b208c-13d3-441b-a199-0387bad7b361 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:37.478442 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 509/1001] 10.222.0.42 () {56 vars in 1112 bytes} [Thu Sep 15 07:21:37 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:21:37.479636 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ec763f-90fc-4397-83c9-76280c585234 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:37.479931 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ec763f-90fc-4397-83c9-76280c585234 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:37.480274 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ec763f-90fc-4397-83c9-76280c585234 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:37.481675 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 510/1002] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:21:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:21:37.481997 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13647f9-8bdf-4ebb-b709-e81e8dd2540e None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:37.487879 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ec763f-90fc-4397-83c9-76280c585234 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:37.517868 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a844ef-d511-43ea-89a7-00e42f3e2b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:37.518496 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a844ef-d511-43ea-89a7-00e42f3e2b61 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:37.518776 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a844ef-d511-43ea-89a7-00e42f3e2b61 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:37.519057 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a844ef-d511-43ea-89a7-00e42f3e2b61 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:37.519640 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a844ef-d511-43ea-89a7-00e42f3e2b61 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:37.521048 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a844ef-d511-43ea-89a7-00e42f3e2b61 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:37.529610 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-11a844ef-d511-43ea-89a7-00e42f3e2b61 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 07:21:37.529610 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:37.529610 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 07:21:37.529610 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 07:21:37.529610 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 07:21:37.529610 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 07:21:37.529610 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 07:21:37.529610 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 07:21:37.529610 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 07:21:37.529610 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:21:37.529610 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 07:21:37.529610 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:21:37.529610 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:37.529610 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:37.529610 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:37.529610 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:37.529610 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:37.530592 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:37.530592 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:37.530592 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:37.530592 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:37.530592 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:37.530592 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:37.530592 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 07:21:37.530592 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 07:21:37.530592 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 07:21:37.530592 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 07:21:37.530592 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:37.530592 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:37.530592 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:37.530592 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:37.530592 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:37.530592 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:37.531428 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:37.531428 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:37.531428 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:37.531428 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:37.531428 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:37.531428 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:37.531428 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:37.531428 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:37.531428 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:37.531428 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:37.531428 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 07:21:37.531428 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 07:21:37.531428 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 07:21:37.531428 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:21:37.531428 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 493/1003] 10.222.0.42 () {56 vars in 1126 bytes} [Thu Sep 15 07:21:37 2022] GET /identity/v3/services/object-store => generated 93 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 07:21:37.532829 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0604d6-f969-4dd1-87ec-d51f19a551f4 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:37.532829 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0604d6-f969-4dd1-87ec-d51f19a551f4 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:37.532953 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0604d6-f969-4dd1-87ec-d51f19a551f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:37.551801 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-6b0604d6-f969-4dd1-87ec-d51f19a551f4 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:21:37.553215 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-6b0604d6-f969-4dd1-87ec-d51f19a551f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:21:37.556388 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:37.577296 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:37.591616 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ec763f-90fc-4397-83c9-76280c585234 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:37.622446 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 494/1004] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:21:37 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 07:21:37.630804 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0604d6-f969-4dd1-87ec-d51f19a551f4 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:37.630930 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85871f91-3f99-443b-a351-decdc186aa0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:37.630930 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85871f91-3f99-443b-a351-decdc186aa0a None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:37.630930 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85871f91-3f99-443b-a351-decdc186aa0a None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:37.630930 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85871f91-3f99-443b-a351-decdc186aa0a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:37.631555 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85871f91-3f99-443b-a351-decdc186aa0a None admin] RBAC: Adding query filter params (interface=public, service_id=7230ac8cac9b4832a7fd96534536ed12, region_id=RegionOne) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:37.631555 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85871f91-3f99-443b-a351-decdc186aa0a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:37.631555 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85871f91-3f99-443b-a351-decdc186aa0a None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:37.634983 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 511/1005] 10.222.0.42 () {56 vars in 1251 bytes} [Thu Sep 15 07:21:37 2022] GET /identity/v3/endpoints?service_id=7230ac8cac9b4832a7fd96534536ed12&interface=public®ion_id=RegionOne => generated 194 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:37.640280 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85871f91-3f99-443b-a351-decdc186aa0a None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:37.736896 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0604d6-f969-4dd1-87ec-d51f19a551f4 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:37.748026 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85871f91-3f99-443b-a351-decdc186aa0a None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:37.773288 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a573d68c-45c8-4bd3-be57-5fe273698970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:37.773842 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a573d68c-45c8-4bd3-be57-5fe273698970 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:37.774223 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a573d68c-45c8-4bd3-be57-5fe273698970 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:37.778249 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a573d68c-45c8-4bd3-be57-5fe273698970 None admin] PATH_INFO: `/v3/services` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:37.778249 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a573d68c-45c8-4bd3-be57-5fe273698970 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:37.778249 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a573d68c-45c8-4bd3-be57-5fe273698970 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:37.778249 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a573d68c-45c8-4bd3-be57-5fe273698970 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:37.778881 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:37.781901 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 495/1006] 10.222.0.42 () {56 vars in 1122 bytes} [Thu Sep 15 07:21:37 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:37.786317 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2629a1-828b-4850-a5f0-a12f00c9c860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:37.786937 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2629a1-828b-4850-a5f0-a12f00c9c860 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:37.787222 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2629a1-828b-4850-a5f0-a12f00c9c860 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:37.787498 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2629a1-828b-4850-a5f0-a12f00c9c860 None admin] PATH_INFO: `/v3/services` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:37.788020 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a573d68c-45c8-4bd3-be57-5fe273698970 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:37.788120 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2629a1-828b-4850-a5f0-a12f00c9c860 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:37.789455 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2629a1-828b-4850-a5f0-a12f00c9c860 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:37.801630 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 512/1007] 10.222.0.42 () {56 vars in 1087 bytes} [Thu Sep 15 07:21:37 2022] GET /identity/v3/services => generated 1727 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 07:21:37.830324 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:37.890805 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2629a1-828b-4850-a5f0-a12f00c9c860 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:37.920448 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a573d68c-45c8-4bd3-be57-5fe273698970 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:37.971414 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d783cb-9938-4d6f-96ed-a49d3858d697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:37.972057 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d783cb-9938-4d6f-96ed-a49d3858d697 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:37.972306 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d783cb-9938-4d6f-96ed-a49d3858d697 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:37.972581 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d783cb-9938-4d6f-96ed-a49d3858d697 None admin] PATH_INFO: `/v3/services` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:37.973207 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d783cb-9938-4d6f-96ed-a49d3858d697 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:37.973509 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d783cb-9938-4d6f-96ed-a49d3858d697 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:37.974909 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d783cb-9938-4d6f-96ed-a49d3858d697 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:37.980412 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 496/1008] 10.222.0.42 () {56 vars in 1122 bytes} [Thu Sep 15 07:21:37 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:37.986238 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d783cb-9938-4d6f-96ed-a49d3858d697 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:38.014131 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2629a1-828b-4850-a5f0-a12f00c9c860 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:38.053250 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5986d7-4f0a-4493-93be-7e2dcf346dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:38.053903 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5986d7-4f0a-4493-93be-7e2dcf346dc6 None admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:38.054182 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5986d7-4f0a-4493-93be-7e2dcf346dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:38.054450 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5986d7-4f0a-4493-93be-7e2dcf346dc6 None admin] PATH_INFO: `/v3/services` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:38.055109 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5986d7-4f0a-4493-93be-7e2dcf346dc6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:38.056529 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5986d7-4f0a-4493-93be-7e2dcf346dc6 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:38.083201 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:38.097548 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d783cb-9938-4d6f-96ed-a49d3858d697 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:38.131266 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a66f7d-497f-412d-8691-70bef1b1f995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:38.131877 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a66f7d-497f-412d-8691-70bef1b1f995 None admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:38.132155 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a66f7d-497f-412d-8691-70bef1b1f995 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:38.132427 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a66f7d-497f-412d-8691-70bef1b1f995 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:38.133088 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a66f7d-497f-412d-8691-70bef1b1f995 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:38.134464 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a66f7d-497f-412d-8691-70bef1b1f995 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:38.146714 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 513/1009] 10.222.0.42 () {60 vars in 1141 bytes} [Thu Sep 15 07:21:37 2022] POST /identity/v3/services => generated 242 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 07:21:38.217096 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 497/1010] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:21:37 2022] POST /identity/v3/endpoints => generated 325 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:21:40.008206 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3039d3-bd59-40e1-a792-953ea0e25621 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:40.008754 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3039d3-bd59-40e1-a792-953ea0e25621 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:40.008754 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3039d3-bd59-40e1-a792-953ea0e25621 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:40.010207 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 514/1011] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:21:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:21:40.015455 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55e40e0-a1de-4b2d-bc8a-befbf999dcde None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:40.015870 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55e40e0-a1de-4b2d-bc8a-befbf999dcde None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:40.016063 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55e40e0-a1de-4b2d-bc8a-befbf999dcde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:40.034869 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-f55e40e0-a1de-4b2d-bc8a-befbf999dcde None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:21:40.036060 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-f55e40e0-a1de-4b2d-bc8a-befbf999dcde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:21:40.115880 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 498/1012] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:21:40 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 07:21:40.337881 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a83b3c-551b-4d4f-be31-e6f0d22e10d9 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:40.338288 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a83b3c-551b-4d4f-be31-e6f0d22e10d9 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:40.338469 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a83b3c-551b-4d4f-be31-e6f0d22e10d9 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:40.339913 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 515/1013] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:21:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:21:40.344975 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc83c827-8150-4be7-b4ba-f14b8a906473 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:40.345826 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc83c827-8150-4be7-b4ba-f14b8a906473 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:40.346097 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc83c827-8150-4be7-b4ba-f14b8a906473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:40.365350 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-dc83c827-8150-4be7-b4ba-f14b8a906473 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:21:40.366550 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-dc83c827-8150-4be7-b4ba-f14b8a906473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:21:40.386535 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f698de-b9fa-4733-a80b-d998d75a865c None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:40.386822 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f698de-b9fa-4733-a80b-d998d75a865c None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:40.387096 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f698de-b9fa-4733-a80b-d998d75a865c None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:40.388541 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 516/1014] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:21:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 07:21:40.394000 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff9c484-01ed-468d-9be5-58979a095b78 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:40.394283 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff9c484-01ed-468d-9be5-58979a095b78 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:40.394563 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff9c484-01ed-468d-9be5-58979a095b78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:40.413185 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-2ff9c484-01ed-468d-9be5-58979a095b78 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:21:40.414371 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-2ff9c484-01ed-468d-9be5-58979a095b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:21:40.436429 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 499/1015] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:21:40 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 07:21:40.437269 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc83c827-8150-4be7-b4ba-f14b8a906473 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:40.442427 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:40.470601 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:40.489194 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 517/1016] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:21:40 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 07:21:40.540253 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc83c827-8150-4be7-b4ba-f14b8a906473 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:40.575956 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b755f759-49f5-4adb-965d-152eded049b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:40.576584 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b755f759-49f5-4adb-965d-152eded049b2 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:40.576864 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b755f759-49f5-4adb-965d-152eded049b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:40.577135 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b755f759-49f5-4adb-965d-152eded049b2 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:40.577762 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b755f759-49f5-4adb-965d-152eded049b2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:40.579118 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b755f759-49f5-4adb-965d-152eded049b2 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:40.587972 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-b755f759-49f5-4adb-965d-152eded049b2 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 15 07:21:40.587972 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:40.587972 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 07:21:40.587972 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 07:21:40.587972 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 07:21:40.587972 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 07:21:40.587972 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 07:21:40.587972 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 07:21:40.587972 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 15 07:21:40.587972 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:21:40.587972 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 07:21:40.587972 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:21:40.587972 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:40.587972 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:40.587972 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:40.587972 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:40.587972 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:40.588951 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:40.588951 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:40.588951 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:40.588951 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:40.588951 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:40.588951 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:40.588951 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 07:21:40.588951 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 07:21:40.588951 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 07:21:40.588951 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 07:21:40.588951 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:40.588951 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:40.588951 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:40.588951 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:40.588951 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:40.588951 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:40.590244 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:40.590244 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:40.590244 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:40.590244 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:40.590244 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:40.590244 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:40.590244 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:40.590244 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:40.590244 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:40.590244 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:40.590244 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 07:21:40.590244 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 07:21:40.590244 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 15 07:21:40.590244 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:21:40.590244 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 500/1017] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 15 07:21:40 2022] GET /identity/v3/services/compute => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:21:40.595461 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff9c484-01ed-468d-9be5-58979a095b78 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:40.700160 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff9c484-01ed-468d-9be5-58979a095b78 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:40.723834 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b755f759-49f5-4adb-965d-152eded049b2 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:40.736347 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6369ab0d-de0c-41b0-9e40-61b86fd148a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:40.737320 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6369ab0d-de0c-41b0-9e40-61b86fd148a0 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:40.737629 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6369ab0d-de0c-41b0-9e40-61b86fd148a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:40.737914 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6369ab0d-de0c-41b0-9e40-61b86fd148a0 None admin] PATH_INFO: `/v3/services` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:40.738538 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6369ab0d-de0c-41b0-9e40-61b86fd148a0 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:40.738842 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6369ab0d-de0c-41b0-9e40-61b86fd148a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:40.740203 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6369ab0d-de0c-41b0-9e40-61b86fd148a0 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:40.749607 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 518/1018] 10.222.0.42 () {56 vars in 1112 bytes} [Thu Sep 15 07:21:40 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:40.750515 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6369ab0d-de0c-41b0-9e40-61b86fd148a0 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:40.795641 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:40.801269 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:40.832166 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b755f759-49f5-4adb-965d-152eded049b2 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:40.850335 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6369ab0d-de0c-41b0-9e40-61b86fd148a0 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:40.865560 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741fbd45-bbf3-4ff9-87ab-ee0bbc7c2cdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:40.866212 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741fbd45-bbf3-4ff9-87ab-ee0bbc7c2cdf None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:40.866494 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741fbd45-bbf3-4ff9-87ab-ee0bbc7c2cdf None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:40.866769 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741fbd45-bbf3-4ff9-87ab-ee0bbc7c2cdf None admin] PATH_INFO: `/v3/services/image` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:40.867350 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741fbd45-bbf3-4ff9-87ab-ee0bbc7c2cdf None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:40.868700 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741fbd45-bbf3-4ff9-87ab-ee0bbc7c2cdf None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:40.877728 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-741fbd45-bbf3-4ff9-87ab-ee0bbc7c2cdf None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 15 07:21:40.877728 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:40.877728 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 07:21:40.877728 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 07:21:40.877728 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 07:21:40.877728 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 07:21:40.877728 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 07:21:40.877728 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 07:21:40.877728 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 15 07:21:40.877728 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:21:40.877728 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 07:21:40.877728 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:21:40.877728 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:40.877728 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:40.877728 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:40.877728 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:40.877728 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:40.878905 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:40.878905 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:40.878905 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:40.878905 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:40.878905 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:40.878905 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:40.878905 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 07:21:40.878905 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 07:21:40.878905 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 07:21:40.878905 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 07:21:40.878905 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:40.878905 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:40.878905 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:40.878905 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:40.878905 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:40.878905 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:40.879940 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:40.879940 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:40.879940 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:40.879940 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:40.879940 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:40.879940 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:40.879940 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:40.879940 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:40.879940 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:40.879940 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:40.879940 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 07:21:40.879940 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 07:21:40.879940 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 15 07:21:40.879940 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:21:40.879940 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 501/1019] 10.222.0.42 () {56 vars in 1105 bytes} [Thu Sep 15 07:21:40 2022] GET /identity/v3/services/image => generated 86 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 07:21:40.880898 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741fbd45-bbf3-4ff9-87ab-ee0bbc7c2cdf None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:40.886706 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5996c93d-375b-4c05-bccf-f4377f95101a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:40.887309 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5996c93d-375b-4c05-bccf-f4377f95101a None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:40.887590 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5996c93d-375b-4c05-bccf-f4377f95101a None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:40.887864 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5996c93d-375b-4c05-bccf-f4377f95101a None admin] PATH_INFO: `/v3/services` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:40.888487 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5996c93d-375b-4c05-bccf-f4377f95101a None admin] RBAC: Adding query filter params (type=compute) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:40.888793 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5996c93d-375b-4c05-bccf-f4377f95101a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:40.890191 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5996c93d-375b-4c05-bccf-f4377f95101a None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:40.900608 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 519/1020] 10.222.0.42 () {56 vars in 1112 bytes} [Thu Sep 15 07:21:40 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:21:40.900608 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5996c93d-375b-4c05-bccf-f4377f95101a None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:41.002325 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741fbd45-bbf3-4ff9-87ab-ee0bbc7c2cdf None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:41.006462 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5996c93d-375b-4c05-bccf-f4377f95101a None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:41.019774 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:41.030946 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:41.036288 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a18bd6-98b7-424c-9ee7-4bf3fe06632a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:41.036959 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a18bd6-98b7-424c-9ee7-4bf3fe06632a None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:41.037232 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a18bd6-98b7-424c-9ee7-4bf3fe06632a None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:41.037502 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a18bd6-98b7-424c-9ee7-4bf3fe06632a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:41.038633 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a18bd6-98b7-424c-9ee7-4bf3fe06632a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:41.040048 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a18bd6-98b7-424c-9ee7-4bf3fe06632a None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:41.040533 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82006fba-fc99-48c1-b704-390dc1b11f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:41.041146 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82006fba-fc99-48c1-b704-390dc1b11f83 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:41.041430 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82006fba-fc99-48c1-b704-390dc1b11f83 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:41.041636 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 502/1021] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:21:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:41.041747 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82006fba-fc99-48c1-b704-390dc1b11f83 None admin] PATH_INFO: `/v3/services` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:41.042358 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82006fba-fc99-48c1-b704-390dc1b11f83 None admin] RBAC: Adding query filter params (name=image) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:41.042466 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a18bd6-98b7-424c-9ee7-4bf3fe06632a None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:41.042801 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82006fba-fc99-48c1-b704-390dc1b11f83 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:41.044017 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82006fba-fc99-48c1-b704-390dc1b11f83 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:41.049684 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 520/1022] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:21:40 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:41.050826 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82006fba-fc99-48c1-b704-390dc1b11f83 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:41.147306 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a18bd6-98b7-424c-9ee7-4bf3fe06632a None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:41.150130 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82006fba-fc99-48c1-b704-390dc1b11f83 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:41.181205 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8c8ebd-3f60-4e2c-80a1-de36d69a0096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:41.181833 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8c8ebd-3f60-4e2c-80a1-de36d69a0096 None admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:41.182112 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8c8ebd-3f60-4e2c-80a1-de36d69a0096 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:41.182379 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8c8ebd-3f60-4e2c-80a1-de36d69a0096 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:41.183033 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8c8ebd-3f60-4e2c-80a1-de36d69a0096 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:41.184461 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb48abdf-5eb0-48b9-8fb4-f639f2b515fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:41.184577 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8c8ebd-3f60-4e2c-80a1-de36d69a0096 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:41.185074 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb48abdf-5eb0-48b9-8fb4-f639f2b515fb None admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:41.185355 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb48abdf-5eb0-48b9-8fb4-f639f2b515fb None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:41.185661 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb48abdf-5eb0-48b9-8fb4-f639f2b515fb None admin] PATH_INFO: `/v3/projects` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:41.186302 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb48abdf-5eb0-48b9-8fb4-f639f2b515fb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:41.187710 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb48abdf-5eb0-48b9-8fb4-f639f2b515fb None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:41.289903 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 503/1023] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:21:41 2022] POST /identity/v3/endpoints => generated 334 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:21:41.290733 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8c8ebd-3f60-4e2c-80a1-de36d69a0096 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:41.302273 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 521/1024] 10.222.0.42 () {60 vars in 1141 bytes} [Thu Sep 15 07:21:41 2022] POST /identity/v3/projects => generated 311 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:21:41.365287 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:41.424267 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8c8ebd-3f60-4e2c-80a1-de36d69a0096 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:41.465234 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1fece4-4b41-426d-becf-6291437d017b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:41.465911 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1fece4-4b41-426d-becf-6291437d017b None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:41.466195 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1fece4-4b41-426d-becf-6291437d017b None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:41.466472 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1fece4-4b41-426d-becf-6291437d017b None admin] PATH_INFO: `/v3/services` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:41.467138 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1fece4-4b41-426d-becf-6291437d017b None admin] RBAC: Adding query filter params (type=image) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:41.467442 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1fece4-4b41-426d-becf-6291437d017b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:41.468859 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1fece4-4b41-426d-becf-6291437d017b None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:41.474839 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 504/1025] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:21:41 2022] GET /identity/v3/services?type=image => generated 350 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:41.481149 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb48abdf-5eb0-48b9-8fb4-f639f2b515fb None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:41.515869 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:41.597960 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb48abdf-5eb0-48b9-8fb4-f639f2b515fb None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:41.634637 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1105f78-d78c-427f-a7b9-0e229aabe5ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:41.635391 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1105f78-d78c-427f-a7b9-0e229aabe5ea None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:41.635672 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1105f78-d78c-427f-a7b9-0e229aabe5ea None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:41.635940 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1105f78-d78c-427f-a7b9-0e229aabe5ea None admin] PATH_INFO: `/v3/endpoints` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:41.636635 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1105f78-d78c-427f-a7b9-0e229aabe5ea None admin] RBAC: Adding query filter params (interface=public, service_id=af2056620506461d8966425eefa96b90, region_id=RegionOne) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:41.636938 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1105f78-d78c-427f-a7b9-0e229aabe5ea None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:41.638363 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1105f78-d78c-427f-a7b9-0e229aabe5ea None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:41.644494 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 522/1026] 10.222.0.42 () {56 vars in 1251 bytes} [Thu Sep 15 07:21:41 2022] GET /identity/v3/endpoints?service_id=af2056620506461d8966425eefa96b90&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:41.650498 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1fece4-4b41-426d-becf-6291437d017b None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:41.753215 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1fece4-4b41-426d-becf-6291437d017b None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:41.769997 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:41.787734 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e84b553-5601-4ea8-bac3-eee44c56fb39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:41.796351 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e84b553-5601-4ea8-bac3-eee44c56fb39 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:41.796351 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e84b553-5601-4ea8-bac3-eee44c56fb39 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:41.796351 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e84b553-5601-4ea8-bac3-eee44c56fb39 None admin] PATH_INFO: `/v3/services` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:41.796351 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e84b553-5601-4ea8-bac3-eee44c56fb39 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:41.796351 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e84b553-5601-4ea8-bac3-eee44c56fb39 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:41.797538 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 505/1027] 10.222.0.42 () {56 vars in 1087 bytes} [Thu Sep 15 07:21:41 2022] GET /identity/v3/services => generated 1957 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:21:43.617998 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64239890-ee85-4edc-a624-20246bb3d6d2 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:43.618542 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64239890-ee85-4edc-a624-20246bb3d6d2 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:43.618542 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64239890-ee85-4edc-a624-20246bb3d6d2 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:43.619888 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 523/1028] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:21:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:21:43.625316 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af5754a-8d3b-44c0-9c8c-e5eb682bae4f None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:43.625762 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af5754a-8d3b-44c0-9c8c-e5eb682bae4f None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:43.625986 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af5754a-8d3b-44c0-9c8c-e5eb682bae4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:43.629176 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4d714e-3e68-4ac1-ba89-70d75b2c6141 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:43.629460 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4d714e-3e68-4ac1-ba89-70d75b2c6141 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:43.629886 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4d714e-3e68-4ac1-ba89-70d75b2c6141 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:43.631364 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 524/1029] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:21:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 07:21:43.637449 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30add2e-56c3-4597-9a1e-0ddf0df1ccba None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:43.637765 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30add2e-56c3-4597-9a1e-0ddf0df1ccba None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:43.638037 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30add2e-56c3-4597-9a1e-0ddf0df1ccba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:43.646315 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-0af5754a-8d3b-44c0-9c8c-e5eb682bae4f None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:21:43.647513 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-0af5754a-8d3b-44c0-9c8c-e5eb682bae4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:21:43.656764 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-b30add2e-56c3-4597-9a1e-0ddf0df1ccba None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:21:43.657989 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-b30add2e-56c3-4597-9a1e-0ddf0df1ccba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:21:43.724718 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 506/1030] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:21:43 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 07:21:43.737617 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 525/1031] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:21:43 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 07:21:43.952343 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed8ffa3-16ee-48e1-97b9-a4fd33554e3d None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:43.952641 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed8ffa3-16ee-48e1-97b9-a4fd33554e3d None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:43.952917 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed8ffa3-16ee-48e1-97b9-a4fd33554e3d None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:43.954327 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 507/1032] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:21:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:21:43.960243 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42899acb-84f9-4fd1-bee2-d7b71f97872a None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:43.960536 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42899acb-84f9-4fd1-bee2-d7b71f97872a None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:43.960810 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42899acb-84f9-4fd1-bee2-d7b71f97872a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:43.979133 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-42899acb-84f9-4fd1-bee2-d7b71f97872a None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:21:43.980293 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-42899acb-84f9-4fd1-bee2-d7b71f97872a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:21:43.995341 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed8ffa3-16ee-48e1-97b9-a4fd33554e3d None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:43.997217 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:44.046846 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 526/1033] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:21:43 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 07:21:44.047680 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42899acb-84f9-4fd1-bee2-d7b71f97872a None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:44.103971 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed8ffa3-16ee-48e1-97b9-a4fd33554e3d None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:44.117642 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:44.146130 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eae34db-40bc-4117-8856-37134efad5c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:44.147273 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eae34db-40bc-4117-8856-37134efad5c5 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:44.147790 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eae34db-40bc-4117-8856-37134efad5c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:44.148371 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eae34db-40bc-4117-8856-37134efad5c5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:44.160521 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eae34db-40bc-4117-8856-37134efad5c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:21:44.161502 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42899acb-84f9-4fd1-bee2-d7b71f97872a None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:44.165306 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-5eae34db-40bc-4117-8856-37134efad5c5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 15 07:21:44.165306 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:44.165306 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:44.165306 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:44.165306 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:44.165306 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:44.165306 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:44.165306 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:44.165306 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:44.165306 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:44.165306 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:44.165306 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:44.165306 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:21:44.165306 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:21:44.165306 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:21:44.165306 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:21:44.166330 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:21:44.166330 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:21:44.166330 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:21:44.166330 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:21:44.166330 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:44.166330 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:44.166330 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:44.166330 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:44.166330 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:44.166330 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:44.166330 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:44.166330 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:44.166330 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:44.166330 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:44.166330 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:44.166330 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:44.177224 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:44.177224 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:44.177224 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:44.177224 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:44.177224 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:21:44.177224 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:21:44.177224 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:21:44.177224 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:21:44.177224 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:21:44.177224 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:21:44.177224 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 15 07:21:44.177224 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:21:44.177224 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 508/1034] 10.222.0.42 () {56 vars in 1120 bytes} [Thu Sep 15 07:21:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 07:21:44.177224 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eae34db-40bc-4117-8856-37134efad5c5 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:44.200817 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d0b9d0-4e58-4014-af43-d03e7a7d3c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:44.200817 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d0b9d0-4e58-4014-af43-d03e7a7d3c44 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:44.200817 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d0b9d0-4e58-4014-af43-d03e7a7d3c44 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:44.201208 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d0b9d0-4e58-4014-af43-d03e7a7d3c44 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:44.202043 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d0b9d0-4e58-4014-af43-d03e7a7d3c44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:44.203683 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d0b9d0-4e58-4014-af43-d03e7a7d3c44 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:44.205451 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 527/1035] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:21:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:44.211324 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d0b9d0-4e58-4014-af43-d03e7a7d3c44 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:44.234676 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:44.290338 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eae34db-40bc-4117-8856-37134efad5c5 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:44.322227 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d0b9d0-4e58-4014-af43-d03e7a7d3c44 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:44.326624 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1fbc70-845c-408f-a3b8-93596e6a3d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:44.327429 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1fbc70-845c-408f-a3b8-93596e6a3d88 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:44.327708 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1fbc70-845c-408f-a3b8-93596e6a3d88 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:44.327980 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1fbc70-845c-408f-a3b8-93596e6a3d88 None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:44.328641 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1fbc70-845c-408f-a3b8-93596e6a3d88 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:44.328945 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1fbc70-845c-408f-a3b8-93596e6a3d88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:44.330601 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1fbc70-845c-408f-a3b8-93596e6a3d88 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:44.344892 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 509/1036] 10.222.0.42 () {56 vars in 1115 bytes} [Thu Sep 15 07:21:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:21:44.344892 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1fbc70-845c-408f-a3b8-93596e6a3d88 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:44.357846 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3595ff03-f838-4d42-96c1-82af1efe51d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:44.358461 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3595ff03-f838-4d42-96c1-82af1efe51d3 None admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:44.358744 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3595ff03-f838-4d42-96c1-82af1efe51d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:44.359018 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3595ff03-f838-4d42-96c1-82af1efe51d3 None admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:44.359672 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3595ff03-f838-4d42-96c1-82af1efe51d3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:44.361768 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3595ff03-f838-4d42-96c1-82af1efe51d3 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:44.372698 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:44.438931 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1fbc70-845c-408f-a3b8-93596e6a3d88 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:44.472776 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399aa0b4-0545-49b9-83b7-cf0818162aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:44.473321 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399aa0b4-0545-49b9-83b7-cf0818162aba None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:44.473622 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399aa0b4-0545-49b9-83b7-cf0818162aba None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:44.473894 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399aa0b4-0545-49b9-83b7-cf0818162aba None admin] PATH_INFO: `/v3/services/image` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:44.474459 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399aa0b4-0545-49b9-83b7-cf0818162aba None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:44.475798 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399aa0b4-0545-49b9-83b7-cf0818162aba None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:44.484327 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-399aa0b4-0545-49b9-83b7-cf0818162aba None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 15 07:21:44.484327 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:44.484327 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 07:21:44.484327 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 07:21:44.484327 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 07:21:44.484327 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 07:21:44.484327 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 07:21:44.484327 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 07:21:44.484327 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 15 07:21:44.484327 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:21:44.484327 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 07:21:44.484327 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:21:44.484327 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:44.484327 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:44.484327 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:44.484327 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:44.484327 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:44.485475 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:44.485475 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:44.485475 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:44.485475 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:44.485475 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:44.485475 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:44.485475 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 07:21:44.485475 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 07:21:44.485475 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 07:21:44.485475 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 07:21:44.485475 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:44.485475 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:44.485475 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:44.485475 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:44.485475 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:44.485475 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:44.487151 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:44.487151 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:44.487151 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:44.487151 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:44.487151 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:44.487151 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:44.487151 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:44.487151 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:44.487151 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:44.487151 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:44.487151 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 07:21:44.487151 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 07:21:44.487151 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 15 07:21:44.487151 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:21:44.487151 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 510/1037] 10.222.0.42 () {56 vars in 1105 bytes} [Thu Sep 15 07:21:44 2022] GET /identity/v3/services/image => generated 86 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:21:44.487950 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399aa0b4-0545-49b9-83b7-cf0818162aba None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:44.536152 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 528/1038] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Sep 15 07:21:44 2022] POST /identity/v3/users => generated 285 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 07:21:44.536990 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3595ff03-f838-4d42-96c1-82af1efe51d3 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:44.601195 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:44.608841 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399aa0b4-0545-49b9-83b7-cf0818162aba None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:44.638742 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:44.653816 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a685a46c-482f-403c-a582-1290e66db6a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:44.654410 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a685a46c-482f-403c-a582-1290e66db6a3 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:44.654692 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a685a46c-482f-403c-a582-1290e66db6a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:44.654990 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a685a46c-482f-403c-a582-1290e66db6a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:44.667040 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a685a46c-482f-403c-a582-1290e66db6a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:44.668514 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a685a46c-482f-403c-a582-1290e66db6a3 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:44.669400 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3595ff03-f838-4d42-96c1-82af1efe51d3 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:44.686395 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-a685a46c-482f-403c-a582-1290e66db6a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:21:44.686395 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:44.686395 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:44.686395 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:44.686395 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:44.686395 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:44.686395 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:44.686395 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:44.686395 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:44.686395 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:44.686395 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:44.686395 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:44.686395 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 07:21:44.686395 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 07:21:44.686395 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 07:21:44.686395 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 07:21:44.687511 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:44.687511 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:44.687511 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:44.687511 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:44.687511 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:44.687511 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:44.687511 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:44.687511 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:44.687511 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:44.687511 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:44.687511 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:44.687511 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:44.687511 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:44.687511 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:44.687511 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:44.687511 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:44.687511 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 07:21:44.689116 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 07:21:44.689116 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:21:44.689116 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:21:44.689116 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 511/1039] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:21:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:21:44.696156 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a685a46c-482f-403c-a582-1290e66db6a3 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:44.712511 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe87fe75-0441-4677-a8c9-f83b0ba95713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:44.713131 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe87fe75-0441-4677-a8c9-f83b0ba95713 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:44.713465 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe87fe75-0441-4677-a8c9-f83b0ba95713 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:44.713776 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe87fe75-0441-4677-a8c9-f83b0ba95713 None admin] PATH_INFO: `/v3/services` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:44.715073 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe87fe75-0441-4677-a8c9-f83b0ba95713 None admin] RBAC: Adding query filter params (name=image) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:44.715382 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe87fe75-0441-4677-a8c9-f83b0ba95713 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:44.716754 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe87fe75-0441-4677-a8c9-f83b0ba95713 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:44.724881 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 529/1040] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:21:44 2022] GET /identity/v3/services?name=image => generated 122 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:44.730581 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe87fe75-0441-4677-a8c9-f83b0ba95713 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:44.812408 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a685a46c-482f-403c-a582-1290e66db6a3 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:44.849009 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe87fe75-0441-4677-a8c9-f83b0ba95713 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:44.849486 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7472b46e-9dcd-4dd6-a39f-981b239b4b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:44.850140 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7472b46e-9dcd-4dd6-a39f-981b239b4b5e None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:44.850429 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7472b46e-9dcd-4dd6-a39f-981b239b4b5e None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:44.850852 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7472b46e-9dcd-4dd6-a39f-981b239b4b5e None admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:44.852281 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7472b46e-9dcd-4dd6-a39f-981b239b4b5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:44.852597 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7472b46e-9dcd-4dd6-a39f-981b239b4b5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:44.853996 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7472b46e-9dcd-4dd6-a39f-981b239b4b5e None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:44.873275 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 512/1041] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 15 07:21:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:21:44.874742 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7472b46e-9dcd-4dd6-a39f-981b239b4b5e None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:44.894477 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:44.897628 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c323c5-2f3d-4aa3-ad41-664fdc292b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:44.898639 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c323c5-2f3d-4aa3-ad41-664fdc292b66 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:44.899190 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c323c5-2f3d-4aa3-ad41-664fdc292b66 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:44.899731 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c323c5-2f3d-4aa3-ad41-664fdc292b66 None admin] PATH_INFO: `/v3/services` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:44.900875 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c323c5-2f3d-4aa3-ad41-664fdc292b66 None admin] RBAC: Adding query filter params (type=image) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:44.901470 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c323c5-2f3d-4aa3-ad41-664fdc292b66 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:44.903803 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c323c5-2f3d-4aa3-ad41-664fdc292b66 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:44.911898 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 530/1042] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:21:44 2022] GET /identity/v3/services?type=image => generated 350 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:21:44.917509 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c323c5-2f3d-4aa3-ad41-664fdc292b66 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:44.921272 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:45.000847 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7472b46e-9dcd-4dd6-a39f-981b239b4b5e None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:45.034364 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c323c5-2f3d-4aa3-ad41-664fdc292b66 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:45.038053 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b6f603-c60a-4cb2-a164-4d91739b9e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:45.038692 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b6f603-c60a-4cb2-a164-4d91739b9e06 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:45.038973 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b6f603-c60a-4cb2-a164-4d91739b9e06 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:45.039246 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b6f603-c60a-4cb2-a164-4d91739b9e06 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:45.046434 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b6f603-c60a-4cb2-a164-4d91739b9e06 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:45.047867 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b6f603-c60a-4cb2-a164-4d91739b9e06 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:45.060044 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-e0b6f603-c60a-4cb2-a164-4d91739b9e06 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 15 07:21:45.060044 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:45.060044 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:45.060044 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:45.060044 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:45.060044 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:45.060044 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:45.060044 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:45.060044 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:45.060044 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:45.060044 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:45.060044 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:45.060044 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 07:21:45.060044 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 07:21:45.060044 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 07:21:45.060044 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 07:21:45.061204 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:45.061204 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:45.061204 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 07:21:45.061204 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 07:21:45.061204 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 07:21:45.061204 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 07:21:45.061204 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:45.061204 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:45.061204 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:45.061204 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:45.061204 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:45.061204 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:45.061204 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:45.061204 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:45.061204 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:45.061204 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:45.061204 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:45.061204 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:45.062223 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:45.062223 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:45.062223 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 07:21:45.062223 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 07:21:45.062223 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 07:21:45.062223 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 07:21:45.062223 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 07:21:45.062223 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 07:21:45.062223 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 15 07:21:45.062223 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:21:45.062223 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 513/1043] 10.222.0.42 () {56 vars in 1093 bytes} [Thu Sep 15 07:21:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 07:21:45.067704 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b6f603-c60a-4cb2-a164-4d91739b9e06 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:45.075437 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f818186-6a04-4a29-8061-e97bd390803b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:45.076054 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f818186-6a04-4a29-8061-e97bd390803b None admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:45.076334 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f818186-6a04-4a29-8061-e97bd390803b None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:45.076664 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f818186-6a04-4a29-8061-e97bd390803b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:45.077373 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f818186-6a04-4a29-8061-e97bd390803b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:45.078826 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f818186-6a04-4a29-8061-e97bd390803b None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:45.135625 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:45.152343 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 531/1044] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:21:44 2022] POST /identity/v3/endpoints => generated 327 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:21:45.182043 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b6f603-c60a-4cb2-a164-4d91739b9e06 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:45.228801 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87620807-2d0e-46c9-9e1b-cb409a90e15c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:45.229420 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87620807-2d0e-46c9-9e1b-cb409a90e15c None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:45.229733 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87620807-2d0e-46c9-9e1b-cb409a90e15c None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:45.230005 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87620807-2d0e-46c9-9e1b-cb409a90e15c None admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:45.230742 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87620807-2d0e-46c9-9e1b-cb409a90e15c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:45.231046 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87620807-2d0e-46c9-9e1b-cb409a90e15c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:45.232497 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87620807-2d0e-46c9-9e1b-cb409a90e15c None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:45.253434 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 514/1045] 10.222.0.42 () {56 vars in 1133 bytes} [Thu Sep 15 07:21:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:21:45.259069 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f818186-6a04-4a29-8061-e97bd390803b None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:45.326080 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:45.371980 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f818186-6a04-4a29-8061-e97bd390803b None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:45.409638 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65285470-f814-4d18-8d75-0afbbd3fa355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:45.410594 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65285470-f814-4d18-8d75-0afbbd3fa355 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:45.411071 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65285470-f814-4d18-8d75-0afbbd3fa355 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:45.411528 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65285470-f814-4d18-8d75-0afbbd3fa355 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:45.482093 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65285470-f814-4d18-8d75-0afbbd3fa355 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:45.483837 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65285470-f814-4d18-8d75-0afbbd3fa355 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:45.547895 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:45.596168 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 532/1046] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 15 07:21:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 07:21:45.597830 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87620807-2d0e-46c9-9e1b-cb409a90e15c None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:45.698911 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87620807-2d0e-46c9-9e1b-cb409a90e15c None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:45.721875 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:45.733620 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac1d584-1a07-4220-92a2-d336db684252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:45.734338 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac1d584-1a07-4220-92a2-d336db684252 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:45.734614 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac1d584-1a07-4220-92a2-d336db684252 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:45.734882 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac1d584-1a07-4220-92a2-d336db684252 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:45.744834 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac1d584-1a07-4220-92a2-d336db684252 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:45.746375 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac1d584-1a07-4220-92a2-d336db684252 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:45.761948 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-4ac1d584-1a07-4220-92a2-d336db684252 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:21:45.761948 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:45.761948 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:45.761948 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:45.761948 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:45.761948 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:45.761948 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:45.761948 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:45.761948 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:45.761948 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:45.761948 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:45.761948 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:45.761948 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 07:21:45.761948 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 07:21:45.761948 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 07:21:45.761948 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 07:21:45.762946 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:45.762946 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:45.762946 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:45.762946 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:45.762946 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:45.762946 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:45.762946 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:45.762946 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:45.762946 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:45.762946 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:45.762946 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:45.762946 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:45.762946 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:45.762946 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:45.762946 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:45.762946 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:45.762946 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 07:21:45.768878 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 07:21:45.768878 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:21:45.768878 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:21:45.768878 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 515/1047] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:21:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 07:21:45.770874 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65285470-f814-4d18-8d75-0afbbd3fa355 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:45.880536 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65285470-f814-4d18-8d75-0afbbd3fa355 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:45.917119 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b764cfb8-05f3-4330-b556-5d347481a552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:45.917766 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b764cfb8-05f3-4330-b556-5d347481a552 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:45.918049 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b764cfb8-05f3-4330-b556-5d347481a552 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:45.918327 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b764cfb8-05f3-4330-b556-5d347481a552 None admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:45.918971 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b764cfb8-05f3-4330-b556-5d347481a552 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:45.919281 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b764cfb8-05f3-4330-b556-5d347481a552 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:45.920647 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b764cfb8-05f3-4330-b556-5d347481a552 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:45.932752 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 533/1048] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 15 07:21:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:45.941749 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac1d584-1a07-4220-92a2-d336db684252 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:46.044993 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac1d584-1a07-4220-92a2-d336db684252 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:46.081076 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193b8bf6-3e22-4bcc-adae-e6c1df5f4e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:46.081768 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193b8bf6-3e22-4bcc-adae-e6c1df5f4e1f None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:46.082056 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193b8bf6-3e22-4bcc-adae-e6c1df5f4e1f None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:46.082336 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193b8bf6-3e22-4bcc-adae-e6c1df5f4e1f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:46.089800 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193b8bf6-3e22-4bcc-adae-e6c1df5f4e1f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:46.091243 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193b8bf6-3e22-4bcc-adae-e6c1df5f4e1f None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:46.105194 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-193b8bf6-3e22-4bcc-adae-e6c1df5f4e1f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 07:21:46.105194 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:46.105194 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:46.105194 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:46.105194 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:46.105194 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:46.105194 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:46.105194 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:46.105194 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:46.105194 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:46.105194 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:46.105194 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:46.105194 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 07:21:46.105194 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 07:21:46.105194 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 07:21:46.106120 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 07:21:46.106120 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:46.106120 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:46.106120 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:46.106120 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:46.106120 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:46.106120 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:46.106120 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:46.106120 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:46.106120 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:46.106120 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:46.106120 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:46.106120 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:46.106120 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:46.106120 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:46.106120 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:46.106120 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:46.107027 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 07:21:46.107027 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 07:21:46.107027 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 07:21:46.107027 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 07:21:46.107027 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 07:21:46.107027 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 07:21:46.107027 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 07:21:46.107027 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:21:46.107027 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 516/1049] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 15 07:21:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 07:21:46.114866 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b764cfb8-05f3-4330-b556-5d347481a552 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:46.140838 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:46.223429 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b764cfb8-05f3-4330-b556-5d347481a552 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:46.280848 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319ae8f3-c05e-445f-bfa3-4a4945eceee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:46.281457 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319ae8f3-c05e-445f-bfa3-4a4945eceee6 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:46.281771 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319ae8f3-c05e-445f-bfa3-4a4945eceee6 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:46.282048 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319ae8f3-c05e-445f-bfa3-4a4945eceee6 None admin] PATH_INFO: `/v3/projects` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:46.282768 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319ae8f3-c05e-445f-bfa3-4a4945eceee6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:46.283583 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319ae8f3-c05e-445f-bfa3-4a4945eceee6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:46.285007 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319ae8f3-c05e-445f-bfa3-4a4945eceee6 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:46.297974 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 534/1050] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 15 07:21:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:21:46.305019 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193b8bf6-3e22-4bcc-adae-e6c1df5f4e1f None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:46.341891 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:46.410235 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193b8bf6-3e22-4bcc-adae-e6c1df5f4e1f None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:46.446402 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4137e1-a1aa-44b2-84a3-9b3e63ce0a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:46.447009 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4137e1-a1aa-44b2-84a3-9b3e63ce0a0b None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:46.447286 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4137e1-a1aa-44b2-84a3-9b3e63ce0a0b None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:46.447557 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4137e1-a1aa-44b2-84a3-9b3e63ce0a0b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:46.448279 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4137e1-a1aa-44b2-84a3-9b3e63ce0a0b None admin] RBAC: Adding query filter params (role.id=0888721641c54c3684692df3615ea1b0, scope.project.id=4e9987720930473da840579654311bc2, user.id=a080c7ab9b684abca6828067f2572c3b) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:46.448585 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4137e1-a1aa-44b2-84a3-9b3e63ce0a0b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:46.450041 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4137e1-a1aa-44b2-84a3-9b3e63ce0a0b None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:46.455964 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 517/1051] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 15 07:21:46 2022] GET /identity/v3/role_assignments?user.id=a080c7ab9b684abca6828067f2572c3b&scope.project.id=4e9987720930473da840579654311bc2&role.id=0888721641c54c3684692df3615ea1b0 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:46.800669 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f0f854-2ece-4b17-aae7-475c778cf9e9 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:46.801204 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f0f854-2ece-4b17-aae7-475c778cf9e9 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:46.801204 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f0f854-2ece-4b17-aae7-475c778cf9e9 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:46.802740 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 535/1052] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:21:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:21:46.807837 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85e6946-39c7-4795-a82a-95fe397e7d59 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:46.808219 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85e6946-39c7-4795-a82a-95fe397e7d59 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:46.808414 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85e6946-39c7-4795-a82a-95fe397e7d59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:46.827354 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-e85e6946-39c7-4795-a82a-95fe397e7d59 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:21:46.828525 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-e85e6946-39c7-4795-a82a-95fe397e7d59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:21:46.896563 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 518/1053] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:21:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 07:21:47.162676 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f0f854-2ece-4b17-aae7-475c778cf9e9 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:47.254912 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:47.276572 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f0f854-2ece-4b17-aae7-475c778cf9e9 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:47.316027 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f85a30c-7910-41c7-8c7a-282927392d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:47.316759 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f85a30c-7910-41c7-8c7a-282927392d15 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:47.317042 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f85a30c-7910-41c7-8c7a-282927392d15 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:47.317319 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f85a30c-7910-41c7-8c7a-282927392d15 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:47.328198 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f85a30c-7910-41c7-8c7a-282927392d15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:21:47.333090 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-6f85a30c-7910-41c7-8c7a-282927392d15 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 07:21:47.333090 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:47.333090 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:47.333090 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:47.333090 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:47.333090 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:47.333090 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:47.333090 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:47.333090 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:47.333090 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:47.333090 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:47.333090 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:47.333090 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:21:47.333090 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:21:47.333090 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:21:47.333090 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:21:47.334224 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:21:47.334224 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:21:47.334224 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:21:47.334224 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:21:47.334224 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:47.334224 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:47.334224 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:47.334224 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:47.334224 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:47.334224 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:47.334224 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:47.334224 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:47.334224 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:47.334224 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:47.334224 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:47.334224 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:47.335065 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:47.335065 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:47.335065 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:47.335065 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:47.335065 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:21:47.335065 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:21:47.335065 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:21:47.335065 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:21:47.335065 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:21:47.335065 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:21:47.335065 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 07:21:47.335065 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:21:47.335065 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 536/1054] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 15 07:21:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:21:47.340086 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85e6946-39c7-4795-a82a-95fe397e7d59 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:47.369848 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:47.393408 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eed07c-3efc-4a70-84ff-94d0bae4b909 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:47.393738 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eed07c-3efc-4a70-84ff-94d0bae4b909 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:47.394003 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eed07c-3efc-4a70-84ff-94d0bae4b909 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:47.395603 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 537/1055] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:21:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:21:47.400374 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d063610b-f5dd-429f-ba0a-afb8e1af3af9 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:47.400664 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d063610b-f5dd-429f-ba0a-afb8e1af3af9 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:47.400946 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d063610b-f5dd-429f-ba0a-afb8e1af3af9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:47.418990 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-d063610b-f5dd-429f-ba0a-afb8e1af3af9 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:21:47.420117 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-d063610b-f5dd-429f-ba0a-afb8e1af3af9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:21:47.447616 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85e6946-39c7-4795-a82a-95fe397e7d59 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:47.483831 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d630d764-fd01-49d4-baae-3f1f4968efb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:47.484439 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d630d764-fd01-49d4-baae-3f1f4968efb9 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:47.484717 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d630d764-fd01-49d4-baae-3f1f4968efb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:47.484993 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d630d764-fd01-49d4-baae-3f1f4968efb9 None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:47.485675 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d630d764-fd01-49d4-baae-3f1f4968efb9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:47.486563 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d630d764-fd01-49d4-baae-3f1f4968efb9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:47.488017 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d630d764-fd01-49d4-baae-3f1f4968efb9 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:47.501116 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 519/1056] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 15 07:21:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:21:47.501599 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 538/1057] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:21:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 07:21:47.502538 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d630d764-fd01-49d4-baae-3f1f4968efb9 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:47.602335 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:47.612211 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d630d764-fd01-49d4-baae-3f1f4968efb9 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:47.647458 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9951bd1-1e5e-40d7-b2c5-73ad91c89536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:47.648097 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9951bd1-1e5e-40d7-b2c5-73ad91c89536 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:47.648380 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9951bd1-1e5e-40d7-b2c5-73ad91c89536 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:47.648654 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9951bd1-1e5e-40d7-b2c5-73ad91c89536 None admin] PATH_INFO: `/v3/users/fa7775e61bb149d09c4e093f6b19d0c4` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:47.662849 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9951bd1-1e5e-40d7-b2c5-73ad91c89536 None admin] RBAC: Authorizing `identity:get_user(user_id=fa7775e61bb149d09c4e093f6b19d0c4)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:47.664278 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9951bd1-1e5e-40d7-b2c5-73ad91c89536 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:47.670484 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 520/1058] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 15 07:21:47 2022] GET /identity/v3/users/fa7775e61bb149d09c4e093f6b19d0c4 => generated 285 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:21:47.671645 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d063610b-f5dd-429f-ba0a-afb8e1af3af9 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:47.742576 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:47.761730 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9951bd1-1e5e-40d7-b2c5-73ad91c89536 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:47.774673 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d063610b-f5dd-429f-ba0a-afb8e1af3af9 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:47.810553 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1b8a5d-e8a0-4ca0-af31-dbd3a8e5d296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:47.811212 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1b8a5d-e8a0-4ca0-af31-dbd3a8e5d296 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:47.811436 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1b8a5d-e8a0-4ca0-af31-dbd3a8e5d296 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:47.811709 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1b8a5d-e8a0-4ca0-af31-dbd3a8e5d296 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:47.867911 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9951bd1-1e5e-40d7-b2c5-73ad91c89536 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:47.884345 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1b8a5d-e8a0-4ca0-af31-dbd3a8e5d296 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:47.886253 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1b8a5d-e8a0-4ca0-af31-dbd3a8e5d296 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:47.902718 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b40b39e-1929-4026-a50e-4720daea8a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:47.903327 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b40b39e-1929-4026-a50e-4720daea8a4b None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:47.903613 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b40b39e-1929-4026-a50e-4720daea8a4b None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:47.903889 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b40b39e-1929-4026-a50e-4720daea8a4b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:47.922034 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:47.963085 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:47.977464 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b40b39e-1929-4026-a50e-4720daea8a4b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:47.979275 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b40b39e-1929-4026-a50e-4720daea8a4b None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:47.996696 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 539/1059] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 15 07:21:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 15 07:21:48.003016 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1b8a5d-e8a0-4ca0-af31-dbd3a8e5d296 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:48.091710 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 521/1060] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 15 07:21:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 15 07:21:48.098714 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b40b39e-1929-4026-a50e-4720daea8a4b None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:48.115892 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1b8a5d-e8a0-4ca0-af31-dbd3a8e5d296 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:48.158954 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a5b1e9-3c75-4e4a-aee4-bd1d656c06e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:48.159985 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a5b1e9-3c75-4e4a-aee4-bd1d656c06e5 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:48.160475 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a5b1e9-3c75-4e4a-aee4-bd1d656c06e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:48.160896 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a5b1e9-3c75-4e4a-aee4-bd1d656c06e5 None admin] PATH_INFO: `/v3/projects/2c7fb9b7cb914824b1a96c70d0653dc0` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:48.162192 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a5b1e9-3c75-4e4a-aee4-bd1d656c06e5 None admin] RBAC: Authorizing `identity:get_project(project_id=2c7fb9b7cb914824b1a96c70d0653dc0)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:48.162914 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:48.164934 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a5b1e9-3c75-4e4a-aee4-bd1d656c06e5 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:48.171950 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 540/1061] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 15 07:21:48 2022] GET /identity/v3/projects/2c7fb9b7cb914824b1a96c70d0653dc0 => generated 311 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:21:48.179109 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a5b1e9-3c75-4e4a-aee4-bd1d656c06e5 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:48.219291 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b40b39e-1929-4026-a50e-4720daea8a4b None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:48.247790 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:48.255533 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a4cc47-fd6a-498d-a1c5-f701bf9e75d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:48.256174 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a4cc47-fd6a-498d-a1c5-f701bf9e75d0 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:48.256454 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a4cc47-fd6a-498d-a1c5-f701bf9e75d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:48.256726 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a4cc47-fd6a-498d-a1c5-f701bf9e75d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:48.267940 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a4cc47-fd6a-498d-a1c5-f701bf9e75d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:48.269381 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a4cc47-fd6a-498d-a1c5-f701bf9e75d0 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:48.284827 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-23a4cc47-fd6a-498d-a1c5-f701bf9e75d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:21:48.284827 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:48.284827 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:48.284827 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:48.284827 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:48.284827 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:48.284827 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:48.284827 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:48.284827 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:48.284827 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:48.284827 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:48.284827 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:48.284827 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 07:21:48.284827 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 07:21:48.284827 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 07:21:48.284827 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 07:21:48.285816 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:48.285816 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:48.285816 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:48.285816 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:48.285816 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:48.285816 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:48.285816 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:48.285816 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:48.285816 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:48.285816 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:48.285816 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:48.285816 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:48.285816 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:48.285816 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:48.285816 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:48.285816 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:48.285816 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 07:21:48.286719 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 07:21:48.286719 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:21:48.286719 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:21:48.286719 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 522/1062] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:21:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 07:21:48.291936 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a4cc47-fd6a-498d-a1c5-f701bf9e75d0 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:48.315776 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a5b1e9-3c75-4e4a-aee4-bd1d656c06e5 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:48.356278 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ee894c-bd88-4e4a-8034-27eefd6c1f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:48.356888 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ee894c-bd88-4e4a-8034-27eefd6c1f0a None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:48.357240 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ee894c-bd88-4e4a-8034-27eefd6c1f0a None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:48.357493 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ee894c-bd88-4e4a-8034-27eefd6c1f0a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:48.358235 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ee894c-bd88-4e4a-8034-27eefd6c1f0a None admin] RBAC: Adding query filter params (role.id=6bbb643e0d8d4bc0a9b27c26edf59216, scope.project.id=2c7fb9b7cb914824b1a96c70d0653dc0, user.id=fa7775e61bb149d09c4e093f6b19d0c4) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:48.358548 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ee894c-bd88-4e4a-8034-27eefd6c1f0a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:48.359975 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ee894c-bd88-4e4a-8034-27eefd6c1f0a None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:48.365988 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 541/1063] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 15 07:21:48 2022] GET /identity/v3/role_assignments?user.id=fa7775e61bb149d09c4e093f6b19d0c4&scope.project.id=2c7fb9b7cb914824b1a96c70d0653dc0&role.id=6bbb643e0d8d4bc0a9b27c26edf59216 => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:21:48.399671 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:48.402980 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a4cc47-fd6a-498d-a1c5-f701bf9e75d0 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:48.442310 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b084bbc6-c529-4289-8831-93e114d0dc70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:48.442844 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b084bbc6-c529-4289-8831-93e114d0dc70 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:48.443124 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b084bbc6-c529-4289-8831-93e114d0dc70 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:48.443395 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b084bbc6-c529-4289-8831-93e114d0dc70 None admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:48.444055 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b084bbc6-c529-4289-8831-93e114d0dc70 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:48.444375 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b084bbc6-c529-4289-8831-93e114d0dc70 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:48.445778 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b084bbc6-c529-4289-8831-93e114d0dc70 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:48.465663 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 523/1064] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 15 07:21:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:21:48.613788 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bfcb31-8e8f-4f37-9723-eef752a4b06c None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:48.614153 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bfcb31-8e8f-4f37-9723-eef752a4b06c None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:48.614378 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bfcb31-8e8f-4f37-9723-eef752a4b06c None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:48.615774 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 542/1065] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:21:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:21:48.621568 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785e9d37-3f6b-4ca4-9dcc-e474925b5e90 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:48.622030 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785e9d37-3f6b-4ca4-9dcc-e474925b5e90 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:48.622379 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785e9d37-3f6b-4ca4-9dcc-e474925b5e90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:48.641221 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-785e9d37-3f6b-4ca4-9dcc-e474925b5e90 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:21:48.643150 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-785e9d37-3f6b-4ca4-9dcc-e474925b5e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:21:48.723108 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 524/1066] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:21:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 07:21:48.990105 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bfcb31-8e8f-4f37-9723-eef752a4b06c None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:49.089885 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bfcb31-8e8f-4f37-9723-eef752a4b06c None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:49.091422 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:49.123936 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b335e14-fa9c-4bcb-9b8d-c9d26dbec3b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:49.124565 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b335e14-fa9c-4bcb-9b8d-c9d26dbec3b4 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:49.124842 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b335e14-fa9c-4bcb-9b8d-c9d26dbec3b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:49.125111 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b335e14-fa9c-4bcb-9b8d-c9d26dbec3b4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:49.135723 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b335e14-fa9c-4bcb-9b8d-c9d26dbec3b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:21:49.140427 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-2b335e14-fa9c-4bcb-9b8d-c9d26dbec3b4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 15 07:21:49.140427 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:49.140427 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:49.140427 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:49.140427 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:49.140427 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:49.140427 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:49.140427 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:49.140427 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:49.140427 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:49.140427 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:49.140427 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:49.140427 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:21:49.140427 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:21:49.140427 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:21:49.140427 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:21:49.141539 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:21:49.141539 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:21:49.141539 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:21:49.141539 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:21:49.141539 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:49.141539 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:49.141539 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:49.141539 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:49.141539 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:49.141539 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:49.141539 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:49.141539 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:49.141539 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:49.141539 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:49.141539 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:49.141539 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:49.143059 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:49.143059 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:49.143059 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:49.143059 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:49.143059 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:21:49.143059 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:21:49.143059 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:21:49.143059 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:21:49.143059 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:21:49.143059 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:21:49.143059 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 15 07:21:49.143059 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:21:49.143059 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 543/1067] 10.222.0.42 () {56 vars in 1120 bytes} [Thu Sep 15 07:21:48 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:21:49.148088 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785e9d37-3f6b-4ca4-9dcc-e474925b5e90 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:49.238071 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:49.247295 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785e9d37-3f6b-4ca4-9dcc-e474925b5e90 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:49.282919 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffc40a4-3018-47fe-8489-1f80060d6a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:49.283536 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffc40a4-3018-47fe-8489-1f80060d6a20 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:49.283811 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffc40a4-3018-47fe-8489-1f80060d6a20 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:49.284078 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffc40a4-3018-47fe-8489-1f80060d6a20 None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:49.284740 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffc40a4-3018-47fe-8489-1f80060d6a20 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:49.285040 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffc40a4-3018-47fe-8489-1f80060d6a20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:49.286420 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffc40a4-3018-47fe-8489-1f80060d6a20 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:49.294810 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 525/1068] 10.222.0.42 () {56 vars in 1115 bytes} [Thu Sep 15 07:21:49 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:49.301061 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b335e14-fa9c-4bcb-9b8d-c9d26dbec3b4 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:49.398159 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b335e14-fa9c-4bcb-9b8d-c9d26dbec3b4 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:49.431443 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b55d0b-d552-459a-b402-cf38b3012fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:49.432068 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b55d0b-d552-459a-b402-cf38b3012fca None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:49.432374 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b55d0b-d552-459a-b402-cf38b3012fca None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:49.432649 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b55d0b-d552-459a-b402-cf38b3012fca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:49.443812 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b55d0b-d552-459a-b402-cf38b3012fca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:49.445262 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b55d0b-d552-459a-b402-cf38b3012fca None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:49.459204 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-93b55d0b-d552-459a-b402-cf38b3012fca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:21:49.459204 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:49.459204 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:49.459204 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:49.459204 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:49.459204 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:49.459204 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:49.459204 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:49.459204 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:49.459204 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:49.459204 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:49.459204 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:49.459204 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 07:21:49.459204 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 07:21:49.459204 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 07:21:49.459204 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 07:21:49.460323 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:49.460323 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:49.460323 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:49.460323 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:49.460323 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:49.460323 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:49.460323 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:49.460323 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:49.460323 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:49.460323 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:49.460323 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:49.460323 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:49.460323 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:49.460323 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:49.460323 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:49.460323 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:49.460323 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 07:21:49.461351 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 07:21:49.461351 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:21:49.461351 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:21:49.461351 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 544/1069] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:21:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 07:21:49.465951 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffc40a4-3018-47fe-8489-1f80060d6a20 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:49.562638 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffc40a4-3018-47fe-8489-1f80060d6a20 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:49.596575 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a052af-f817-4ba0-83e0-7fb10969043c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:49.597176 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a052af-f817-4ba0-83e0-7fb10969043c None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:49.597453 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a052af-f817-4ba0-83e0-7fb10969043c None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:49.597757 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a052af-f817-4ba0-83e0-7fb10969043c None admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:49.598390 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a052af-f817-4ba0-83e0-7fb10969043c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:49.598689 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a052af-f817-4ba0-83e0-7fb10969043c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:49.600032 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a052af-f817-4ba0-83e0-7fb10969043c None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:49.611561 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 526/1070] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 15 07:21:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:49.617098 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b55d0b-d552-459a-b402-cf38b3012fca None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:49.634029 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:49.717547 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b55d0b-d552-459a-b402-cf38b3012fca None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:49.750490 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314a3814-43c1-4189-9e06-dcb268926f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:49.751119 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a3814-43c1-4189-9e06-dcb268926f2c None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:49.751399 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a3814-43c1-4189-9e06-dcb268926f2c None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:49.752239 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a3814-43c1-4189-9e06-dcb268926f2c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:49.764680 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314a3814-43c1-4189-9e06-dcb268926f2c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:49.766172 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314a3814-43c1-4189-9e06-dcb268926f2c None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:49.777643 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-314a3814-43c1-4189-9e06-dcb268926f2c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 15 07:21:49.777643 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:49.777643 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:49.777643 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:49.777643 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:49.777643 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:49.777643 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:49.777643 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:49.777643 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:49.777643 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:49.777643 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:49.777643 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:49.777643 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 07:21:49.777643 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 07:21:49.777643 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 07:21:49.777643 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 07:21:49.778648 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:49.778648 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:49.778648 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 07:21:49.778648 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 07:21:49.778648 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 07:21:49.778648 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 07:21:49.778648 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:49.778648 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:49.778648 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:49.778648 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:49.778648 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:49.778648 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:49.778648 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:49.778648 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:49.778648 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:49.778648 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:49.778648 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:49.778648 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:49.779578 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:49.779578 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:49.779578 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 07:21:49.779578 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 07:21:49.779578 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 07:21:49.779578 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 07:21:49.779578 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 07:21:49.779578 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 07:21:49.779578 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 15 07:21:49.779578 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:21:49.779578 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 545/1071] 10.222.0.42 () {56 vars in 1093 bytes} [Thu Sep 15 07:21:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 07:21:49.784618 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a052af-f817-4ba0-83e0-7fb10969043c None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:49.799972 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:49.888594 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a052af-f817-4ba0-83e0-7fb10969043c None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:49.921206 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f86255-9ff8-41c5-b896-f0dcefe97d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:49.921840 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f86255-9ff8-41c5-b896-f0dcefe97d59 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:49.922121 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f86255-9ff8-41c5-b896-f0dcefe97d59 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:49.922391 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f86255-9ff8-41c5-b896-f0dcefe97d59 None admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:49.923109 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f86255-9ff8-41c5-b896-f0dcefe97d59 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:49.923963 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f86255-9ff8-41c5-b896-f0dcefe97d59 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:49.925712 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f86255-9ff8-41c5-b896-f0dcefe97d59 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:49.943382 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 527/1072] 10.222.0.42 () {56 vars in 1133 bytes} [Thu Sep 15 07:21:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:49.949185 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314a3814-43c1-4189-9e06-dcb268926f2c None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:50.026242 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:50.052017 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314a3814-43c1-4189-9e06-dcb268926f2c None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:50.086706 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b04f31-1ced-44b6-9368-b211bb7db5a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:50.087338 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b04f31-1ced-44b6-9368-b211bb7db5a5 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:50.087621 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b04f31-1ced-44b6-9368-b211bb7db5a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:50.087896 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b04f31-1ced-44b6-9368-b211bb7db5a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:50.098750 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b04f31-1ced-44b6-9368-b211bb7db5a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:50.100195 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b04f31-1ced-44b6-9368-b211bb7db5a5 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:50.115030 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-44b04f31-1ced-44b6-9368-b211bb7db5a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:21:50.115030 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:50.115030 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:50.115030 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:50.115030 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:50.115030 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:50.115030 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:50.115030 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:50.115030 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:50.115030 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:50.115030 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:50.115030 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:50.115030 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 07:21:50.115030 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 07:21:50.115030 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 07:21:50.115030 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 07:21:50.116261 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:50.116261 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:50.116261 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:50.116261 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:50.116261 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:50.116261 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:50.116261 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:50.116261 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:50.116261 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:50.116261 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:50.116261 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:50.116261 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:50.116261 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:50.116261 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:50.116261 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:50.116261 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:50.116261 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 07:21:50.117481 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 07:21:50.117481 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:21:50.117481 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:21:50.121621 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 546/1073] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:21:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:21:50.124038 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f86255-9ff8-41c5-b896-f0dcefe97d59 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:50.202466 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:50.233870 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f86255-9ff8-41c5-b896-f0dcefe97d59 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:50.267738 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a089d26-b437-418a-85a6-e30321bd3546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:50.268258 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a089d26-b437-418a-85a6-e30321bd3546 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:50.268536 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a089d26-b437-418a-85a6-e30321bd3546 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:50.268806 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a089d26-b437-418a-85a6-e30321bd3546 None admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:50.269434 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a089d26-b437-418a-85a6-e30321bd3546 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:50.269772 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a089d26-b437-418a-85a6-e30321bd3546 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:50.271115 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a089d26-b437-418a-85a6-e30321bd3546 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:50.281901 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 528/1074] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 15 07:21:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:50.288209 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b04f31-1ced-44b6-9368-b211bb7db5a5 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:50.387558 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b04f31-1ced-44b6-9368-b211bb7db5a5 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:50.413194 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:50.421748 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bbc459-82b5-4e6b-bc05-02a811f6ddaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:50.422380 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bbc459-82b5-4e6b-bc05-02a811f6ddaf None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:50.422657 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bbc459-82b5-4e6b-bc05-02a811f6ddaf None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:50.422925 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bbc459-82b5-4e6b-bc05-02a811f6ddaf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:50.430100 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bbc459-82b5-4e6b-bc05-02a811f6ddaf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:50.444652 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bbc459-82b5-4e6b-bc05-02a811f6ddaf None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:50.444652 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-52bbc459-82b5-4e6b-bc05-02a811f6ddaf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 07:21:50.444652 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:50.444652 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:50.444652 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:50.444652 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:50.444652 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:50.444652 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:50.444652 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:50.444652 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:50.444652 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:50.444652 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:50.444652 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:50.444652 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 07:21:50.445687 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 07:21:50.445687 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 07:21:50.445687 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 07:21:50.445687 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:50.445687 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:50.445687 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:50.445687 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:50.445687 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:50.445687 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:50.445687 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:50.445687 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:50.445687 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:50.445687 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:50.445687 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:50.445687 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:50.445687 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:50.445687 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:50.446801 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:50.446801 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:50.446801 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 07:21:50.446801 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 07:21:50.446801 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 07:21:50.446801 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 07:21:50.446801 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 07:21:50.446801 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 07:21:50.446801 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 07:21:50.446801 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:21:50.446801 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 547/1075] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 15 07:21:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:21:50.450028 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a089d26-b437-418a-85a6-e30321bd3546 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:50.552034 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a089d26-b437-418a-85a6-e30321bd3546 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:50.569731 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb6b2a0-1e93-4746-817a-8a01a333cc59 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:50.570112 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb6b2a0-1e93-4746-817a-8a01a333cc59 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:50.570453 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb6b2a0-1e93-4746-817a-8a01a333cc59 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:50.571872 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 548/1076] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:21:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 07:21:50.576787 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f2e42-3c1a-4c9f-8ebb-7ed548ea2bfc None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:50.577069 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f2e42-3c1a-4c9f-8ebb-7ed548ea2bfc None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:50.577343 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f2e42-3c1a-4c9f-8ebb-7ed548ea2bfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:50.580576 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:50.586279 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c043ddd-325b-4bee-8514-97c5ebadb8e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:50.586882 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c043ddd-325b-4bee-8514-97c5ebadb8e1 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:50.587159 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c043ddd-325b-4bee-8514-97c5ebadb8e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:50.587430 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c043ddd-325b-4bee-8514-97c5ebadb8e1 None admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:50.588111 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c043ddd-325b-4bee-8514-97c5ebadb8e1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:50.588414 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c043ddd-325b-4bee-8514-97c5ebadb8e1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:50.589847 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c043ddd-325b-4bee-8514-97c5ebadb8e1 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:50.596306 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-9a5f2e42-3c1a-4c9f-8ebb-7ed548ea2bfc None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:21:50.597494 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-9a5f2e42-3c1a-4c9f-8ebb-7ed548ea2bfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:21:50.606468 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 529/1077] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 15 07:21:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:21:50.614858 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c043ddd-325b-4bee-8514-97c5ebadb8e1 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:50.665910 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 549/1078] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:21:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 07:21:50.712883 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c043ddd-325b-4bee-8514-97c5ebadb8e1 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:50.747608 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3ea5dc-24c4-43c9-b07c-9c8478ace503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:50.748362 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3ea5dc-24c4-43c9-b07c-9c8478ace503 None admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:50.748644 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3ea5dc-24c4-43c9-b07c-9c8478ace503 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:50.748920 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3ea5dc-24c4-43c9-b07c-9c8478ace503 None admin] PATH_INFO: `/v3/projects/4e9987720930473da840579654311bc2/users/a080c7ab9b684abca6828067f2572c3b/roles/0888721641c54c3684692df3615ea1b0` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:50.751118 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3ea5dc-24c4-43c9-b07c-9c8478ace503 None admin] RBAC: Authorizing `identity:create_grant(project_id=4e9987720930473da840579654311bc2, user_id=a080c7ab9b684abca6828067f2572c3b, role_id=0888721641c54c3684692df3615ea1b0)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:50.752870 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3ea5dc-24c4-43c9-b07c-9c8478ace503 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:50.815626 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 530/1079] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Sep 15 07:21:50 2022] PUT /identity/v3/projects/4e9987720930473da840579654311bc2/users/a080c7ab9b684abca6828067f2572c3b/roles/0888721641c54c3684692df3615ea1b0 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 07:21:50.935242 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5f2e42-3c1a-4c9f-8ebb-7ed548ea2bfc None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:51.038919 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5f2e42-3c1a-4c9f-8ebb-7ed548ea2bfc None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:51.055039 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:51.073467 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecd7569-9221-4378-a3c6-9e7c44eefa13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:51.073467 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecd7569-9221-4378-a3c6-9e7c44eefa13 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:51.073758 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecd7569-9221-4378-a3c6-9e7c44eefa13 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:51.073971 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecd7569-9221-4378-a3c6-9e7c44eefa13 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:51.084388 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecd7569-9221-4378-a3c6-9e7c44eefa13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:21:51.089264 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-3ecd7569-9221-4378-a3c6-9e7c44eefa13 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 07:21:51.089264 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:51.089264 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:51.089264 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:51.089264 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:51.089264 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:51.089264 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:51.089264 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:51.089264 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:51.089264 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:51.089264 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:51.089264 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:51.089264 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:21:51.089264 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:21:51.089264 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:21:51.089264 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:21:51.090399 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:21:51.090399 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:21:51.090399 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:21:51.090399 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:21:51.090399 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:51.090399 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:51.090399 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:51.090399 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:51.090399 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:51.090399 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:51.090399 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:51.090399 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:51.090399 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:51.090399 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:51.090399 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:51.090399 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:51.091518 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:51.091518 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:51.091518 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:51.091518 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:51.091518 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:21:51.091518 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:21:51.091518 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:21:51.091518 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:21:51.091518 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:21:51.091518 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:21:51.091518 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 07:21:51.091518 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:21:51.091518 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 550/1080] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 15 07:21:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:21:51.096521 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3ea5dc-24c4-43c9-b07c-9c8478ace503 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:51.147501 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:51.197193 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3ea5dc-24c4-43c9-b07c-9c8478ace503 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:51.231476 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883c7d74-e56e-4844-b637-f43df5369204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:51.232100 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883c7d74-e56e-4844-b637-f43df5369204 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:51.232341 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883c7d74-e56e-4844-b637-f43df5369204 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:51.232614 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883c7d74-e56e-4844-b637-f43df5369204 None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:51.233315 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883c7d74-e56e-4844-b637-f43df5369204 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:51.233722 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883c7d74-e56e-4844-b637-f43df5369204 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:51.235074 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883c7d74-e56e-4844-b637-f43df5369204 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:51.243758 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 531/1081] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 15 07:21:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:21:51.249944 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecd7569-9221-4378-a3c6-9e7c44eefa13 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:51.350039 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecd7569-9221-4378-a3c6-9e7c44eefa13 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:51.383867 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3ca1df-3025-4825-b480-5754e5c0a9e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:51.384492 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3ca1df-3025-4825-b480-5754e5c0a9e3 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:51.384771 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3ca1df-3025-4825-b480-5754e5c0a9e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:51.385041 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3ca1df-3025-4825-b480-5754e5c0a9e3 None admin] PATH_INFO: `/v3/users/fa7775e61bb149d09c4e093f6b19d0c4` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:51.386160 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3ca1df-3025-4825-b480-5754e5c0a9e3 None admin] RBAC: Authorizing `identity:get_user(user_id=fa7775e61bb149d09c4e093f6b19d0c4)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:51.387587 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3ca1df-3025-4825-b480-5754e5c0a9e3 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:51.393990 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 551/1082] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 15 07:21:51 2022] GET /identity/v3/users/fa7775e61bb149d09c4e093f6b19d0c4 => generated 285 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:51.395120 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883c7d74-e56e-4844-b637-f43df5369204 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:51.491366 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883c7d74-e56e-4844-b637-f43df5369204 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:51.503749 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:51.524650 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aa40f4-698c-4d4b-8948-c66f0bc67ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:51.525277 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aa40f4-698c-4d4b-8948-c66f0bc67ff5 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:51.525694 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aa40f4-698c-4d4b-8948-c66f0bc67ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:51.526015 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aa40f4-698c-4d4b-8948-c66f0bc67ff5 None admin] PATH_INFO: `/v3/projects/2c7fb9b7cb914824b1a96c70d0653dc0` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:51.527130 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1aa40f4-698c-4d4b-8948-c66f0bc67ff5 None admin] RBAC: Authorizing `identity:get_project(project_id=2c7fb9b7cb914824b1a96c70d0653dc0)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:51.528540 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1aa40f4-698c-4d4b-8948-c66f0bc67ff5 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:51.530707 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 532/1083] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 15 07:21:51 2022] GET /identity/v3/projects/2c7fb9b7cb914824b1a96c70d0653dc0 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:51.536833 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3ca1df-3025-4825-b480-5754e5c0a9e3 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:51.582612 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:51.635324 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3ca1df-3025-4825-b480-5754e5c0a9e3 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:51.668958 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c983f88e-8cbf-44d0-8896-4095c6e25464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:51.669644 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c983f88e-8cbf-44d0-8896-4095c6e25464 None admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:51.669922 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c983f88e-8cbf-44d0-8896-4095c6e25464 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:51.670194 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c983f88e-8cbf-44d0-8896-4095c6e25464 None admin] PATH_INFO: `/v3/projects/2c7fb9b7cb914824b1a96c70d0653dc0/users/fa7775e61bb149d09c4e093f6b19d0c4/roles/6bbb643e0d8d4bc0a9b27c26edf59216` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:51.672059 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c983f88e-8cbf-44d0-8896-4095c6e25464 None admin] RBAC: Authorizing `identity:create_grant(project_id=2c7fb9b7cb914824b1a96c70d0653dc0, user_id=fa7775e61bb149d09c4e093f6b19d0c4, role_id=6bbb643e0d8d4bc0a9b27c26edf59216)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:51.673558 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c983f88e-8cbf-44d0-8896-4095c6e25464 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:51.730552 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 552/1084] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Sep 15 07:21:51 2022] PUT /identity/v3/projects/2c7fb9b7cb914824b1a96c70d0653dc0/users/fa7775e61bb149d09c4e093f6b19d0c4/roles/6bbb643e0d8d4bc0a9b27c26edf59216 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:21:53.005390 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1b9792-86a3-445b-9ea4-6964779ab298 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:53.005952 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1b9792-86a3-445b-9ea4-6964779ab298 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:53.005952 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1b9792-86a3-445b-9ea4-6964779ab298 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:53.007336 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 533/1085] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:21:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:21:53.012785 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1903ca28-4486-4bf5-80bb-5a4a75c8b3b7 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:53.013222 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1903ca28-4486-4bf5-80bb-5a4a75c8b3b7 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:53.013346 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1903ca28-4486-4bf5-80bb-5a4a75c8b3b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:53.031687 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-1903ca28-4486-4bf5-80bb-5a4a75c8b3b7 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:21:53.032874 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-1903ca28-4486-4bf5-80bb-5a4a75c8b3b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:21:53.099625 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 553/1086] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:21:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 07:21:53.377428 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1b9792-86a3-445b-9ea4-6964779ab298 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:53.478048 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1b9792-86a3-445b-9ea4-6964779ab298 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:53.512019 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4048cd-f62a-46a2-b9af-49682b54ab92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:53.512680 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4048cd-f62a-46a2-b9af-49682b54ab92 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:53.512965 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4048cd-f62a-46a2-b9af-49682b54ab92 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:53.513245 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4048cd-f62a-46a2-b9af-49682b54ab92 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:53.523907 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4048cd-f62a-46a2-b9af-49682b54ab92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:21:53.528755 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-1b4048cd-f62a-46a2-b9af-49682b54ab92 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 15 07:21:53.528755 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:53.528755 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:53.528755 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:53.528755 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:53.528755 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:53.528755 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:53.528755 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:53.528755 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:53.528755 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:53.528755 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:53.528755 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:53.528755 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:21:53.528755 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:21:53.528755 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:21:53.528755 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:21:53.529738 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:21:53.529738 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:21:53.529738 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:21:53.529738 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:21:53.529738 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:53.529738 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:53.529738 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:53.529738 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:53.529738 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:53.529738 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:53.529738 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:53.529738 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:53.529738 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:53.529738 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:53.529738 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:53.529738 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:53.530560 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:53.530560 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:53.530560 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:53.530560 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:53.530560 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:21:53.530560 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:21:53.530560 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:21:53.530560 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:21:53.530560 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:21:53.530560 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:21:53.530560 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 15 07:21:53.530560 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:21:53.530560 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 534/1087] 10.222.0.42 () {56 vars in 1120 bytes} [Thu Sep 15 07:21:53 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:21:53.535911 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1903ca28-4486-4bf5-80bb-5a4a75c8b3b7 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:53.569841 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:53.641226 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1903ca28-4486-4bf5-80bb-5a4a75c8b3b7 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:53.675680 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abf136a-8061-4696-9eae-fa7593e86843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:53.676286 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf136a-8061-4696-9eae-fa7593e86843 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:53.676573 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf136a-8061-4696-9eae-fa7593e86843 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:53.676846 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf136a-8061-4696-9eae-fa7593e86843 None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:53.677509 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abf136a-8061-4696-9eae-fa7593e86843 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:53.677916 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abf136a-8061-4696-9eae-fa7593e86843 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:53.679285 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abf136a-8061-4696-9eae-fa7593e86843 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:53.687411 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 554/1088] 10.222.0.42 () {56 vars in 1115 bytes} [Thu Sep 15 07:21:53 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:53.694281 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4048cd-f62a-46a2-b9af-49682b54ab92 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:53.737135 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:53.809214 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4048cd-f62a-46a2-b9af-49682b54ab92 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:53.842840 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d0ebc1-94d1-4c9d-89b8-0727b4834874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:53.843484 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d0ebc1-94d1-4c9d-89b8-0727b4834874 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:53.843763 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d0ebc1-94d1-4c9d-89b8-0727b4834874 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:53.844035 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d0ebc1-94d1-4c9d-89b8-0727b4834874 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:53.855906 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d0ebc1-94d1-4c9d-89b8-0727b4834874 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:53.857333 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d0ebc1-94d1-4c9d-89b8-0727b4834874 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:53.867762 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e678df1-2420-41f7-9b91-ab15cc36cbf7 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:53.868049 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e678df1-2420-41f7-9b91-ab15cc36cbf7 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:53.868321 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e678df1-2420-41f7-9b91-ab15cc36cbf7 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:53.869740 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 555/1089] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:21:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 07:21:53.871873 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-a6d0ebc1-94d1-4c9d-89b8-0727b4834874 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:21:53.871873 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:53.871873 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:53.871873 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:53.871873 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:53.871873 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:53.871873 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:53.871873 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:53.871873 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:53.871873 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:53.871873 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:53.871873 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:53.871873 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 07:21:53.871873 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 07:21:53.871873 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 07:21:53.871873 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 07:21:53.872765 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:53.872765 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:53.872765 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:53.872765 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:53.872765 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:53.872765 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:53.872765 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:53.872765 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:53.872765 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:53.872765 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:53.872765 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:53.872765 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:53.872765 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:53.872765 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:53.872765 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:53.872765 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:53.872765 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 07:21:53.873687 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 07:21:53.873687 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:21:53.873687 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:21:53.873687 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 535/1090] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:21:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:21:53.875235 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696ab352-9a2a-47c0-a3b3-7cdf36a423f8 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:53.875526 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696ab352-9a2a-47c0-a3b3-7cdf36a423f8 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:53.875808 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696ab352-9a2a-47c0-a3b3-7cdf36a423f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:53.878442 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d0ebc1-94d1-4c9d-89b8-0727b4834874 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:53.894645 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-696ab352-9a2a-47c0-a3b3-7cdf36a423f8 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:21:53.895771 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-696ab352-9a2a-47c0-a3b3-7cdf36a423f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:21:53.967719 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 556/1091] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:21:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 07:21:53.973965 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:53.982300 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d0ebc1-94d1-4c9d-89b8-0727b4834874 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:54.015837 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59306243-f076-4327-b237-dacd36edb827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:54.016469 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59306243-f076-4327-b237-dacd36edb827 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:54.016751 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59306243-f076-4327-b237-dacd36edb827 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:54.017021 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59306243-f076-4327-b237-dacd36edb827 None admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:54.018704 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59306243-f076-4327-b237-dacd36edb827 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:54.019019 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59306243-f076-4327-b237-dacd36edb827 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:54.020371 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59306243-f076-4327-b237-dacd36edb827 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:54.031949 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 536/1092] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 15 07:21:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:54.037552 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696ab352-9a2a-47c0-a3b3-7cdf36a423f8 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:54.064050 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:54.134676 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696ab352-9a2a-47c0-a3b3-7cdf36a423f8 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:54.172639 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8350c3f5-5bd4-4dfd-9d38-1302947f0bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:54.173205 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8350c3f5-5bd4-4dfd-9d38-1302947f0bbb None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:54.173485 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8350c3f5-5bd4-4dfd-9d38-1302947f0bbb None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:54.174171 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8350c3f5-5bd4-4dfd-9d38-1302947f0bbb None admin] PATH_INFO: `/v3/users/nova` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:54.182724 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8350c3f5-5bd4-4dfd-9d38-1302947f0bbb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:54.184175 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8350c3f5-5bd4-4dfd-9d38-1302947f0bbb None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:54.197294 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-8350c3f5-5bd4-4dfd-9d38-1302947f0bbb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 15 07:21:54.197294 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:54.197294 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:54.197294 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:54.197294 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:54.197294 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:54.197294 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:54.197294 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:54.197294 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:54.197294 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:54.197294 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:54.197294 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:54.197294 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 07:21:54.197294 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 07:21:54.197294 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 07:21:54.197294 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 07:21:54.198413 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:54.198413 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:54.198413 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 07:21:54.198413 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 07:21:54.198413 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 07:21:54.198413 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 07:21:54.198413 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:54.198413 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:54.198413 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:54.198413 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:54.198413 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:54.198413 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:54.198413 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:54.198413 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:54.198413 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:54.198413 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:54.198413 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:54.198413 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:54.199384 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:54.199384 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:54.199384 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 07:21:54.199384 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 07:21:54.199384 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 07:21:54.199384 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 07:21:54.199384 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 07:21:54.199384 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 07:21:54.199384 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 15 07:21:54.199384 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:21:54.199384 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 557/1093] 10.222.0.42 () {56 vars in 1093 bytes} [Thu Sep 15 07:21:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:21:54.204320 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59306243-f076-4327-b237-dacd36edb827 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:54.235369 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8350c3f5-5bd4-4dfd-9d38-1302947f0bbb None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:54.304628 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59306243-f076-4327-b237-dacd36edb827 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:54.324299 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:54.338177 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a46598-afcb-46a7-b674-92f9447bdc53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:54.342039 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a46598-afcb-46a7-b674-92f9447bdc53 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:54.342039 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a46598-afcb-46a7-b674-92f9447bdc53 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:54.342039 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a46598-afcb-46a7-b674-92f9447bdc53 None admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:54.342313 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8350c3f5-5bd4-4dfd-9d38-1302947f0bbb None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:54.342403 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a46598-afcb-46a7-b674-92f9447bdc53 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:54.342403 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a46598-afcb-46a7-b674-92f9447bdc53 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:54.342403 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a46598-afcb-46a7-b674-92f9447bdc53 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:54.359096 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 537/1094] 10.222.0.42 () {56 vars in 1133 bytes} [Thu Sep 15 07:21:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:54.365521 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a46598-afcb-46a7-b674-92f9447bdc53 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:54.374067 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b4583a-e96f-4c5c-b0c5-ca4ae6983a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:54.374730 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b4583a-e96f-4c5c-b0c5-ca4ae6983a7d None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:54.375019 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b4583a-e96f-4c5c-b0c5-ca4ae6983a7d None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:54.375298 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b4583a-e96f-4c5c-b0c5-ca4ae6983a7d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:54.378187 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:54.386192 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b4583a-e96f-4c5c-b0c5-ca4ae6983a7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:21:54.391509 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-58b4583a-e96f-4c5c-b0c5-ca4ae6983a7d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 07:21:54.391509 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:54.391509 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:54.391509 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:54.391509 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:54.391509 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:54.391509 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:54.391509 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:54.391509 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:54.391509 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:54.391509 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:54.391509 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:54.391509 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:21:54.391509 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:21:54.391509 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:21:54.391509 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:21:54.392544 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:21:54.392544 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:21:54.392544 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:21:54.392544 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:21:54.392544 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:54.392544 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:54.392544 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:54.392544 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:54.392544 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:54.392544 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:54.392544 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:54.392544 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:54.392544 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:54.392544 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:54.392544 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:54.392544 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:54.393427 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:54.393427 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:54.393427 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:54.393427 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:54.393427 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:21:54.393427 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:21:54.393427 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:21:54.393427 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:21:54.393427 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:21:54.393427 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:21:54.393427 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 07:21:54.393427 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:21:54.393427 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 558/1095] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 15 07:21:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 07:21:54.398839 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b4583a-e96f-4c5c-b0c5-ca4ae6983a7d None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:54.466136 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a46598-afcb-46a7-b674-92f9447bdc53 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:54.497947 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b4583a-e96f-4c5c-b0c5-ca4ae6983a7d None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:54.499528 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638b1798-8f97-4fa7-9fce-5506a9b491a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:54.500134 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638b1798-8f97-4fa7-9fce-5506a9b491a5 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:54.500418 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638b1798-8f97-4fa7-9fce-5506a9b491a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:54.500695 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638b1798-8f97-4fa7-9fce-5506a9b491a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:54.531679 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae27678-e70b-4858-b444-966eb149b641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:54.532279 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae27678-e70b-4858-b444-966eb149b641 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:54.532561 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae27678-e70b-4858-b444-966eb149b641 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:54.532835 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae27678-e70b-4858-b444-966eb149b641 None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:54.534241 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae27678-e70b-4858-b444-966eb149b641 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:54.534548 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae27678-e70b-4858-b444-966eb149b641 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:54.535921 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae27678-e70b-4858-b444-966eb149b641 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:54.544719 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 559/1096] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 15 07:21:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:21:54.550930 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae27678-e70b-4858-b444-966eb149b641 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:54.561115 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:54.569426 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638b1798-8f97-4fa7-9fce-5506a9b491a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:54.572225 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638b1798-8f97-4fa7-9fce-5506a9b491a5 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:54.582977 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:54.650694 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae27678-e70b-4858-b444-966eb149b641 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:54.672168 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 538/1097] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 15 07:21:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 07:21:54.678756 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638b1798-8f97-4fa7-9fce-5506a9b491a5 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:54.688433 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aa4f11-8600-49f8-be9d-894f28bcb9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:54.690002 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aa4f11-8600-49f8-be9d-894f28bcb9fc None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:54.690283 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aa4f11-8600-49f8-be9d-894f28bcb9fc None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:54.690751 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aa4f11-8600-49f8-be9d-894f28bcb9fc None admin] PATH_INFO: `/v3/users/fa7775e61bb149d09c4e093f6b19d0c4` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:54.691838 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11aa4f11-8600-49f8-be9d-894f28bcb9fc None admin] RBAC: Authorizing `identity:get_user(user_id=fa7775e61bb149d09c4e093f6b19d0c4)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:54.693254 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11aa4f11-8600-49f8-be9d-894f28bcb9fc None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:54.702710 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 560/1098] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 15 07:21:54 2022] GET /identity/v3/users/fa7775e61bb149d09c4e093f6b19d0c4 => generated 285 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:21:54.703619 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aa4f11-8600-49f8-be9d-894f28bcb9fc None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:54.781141 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:54.784999 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638b1798-8f97-4fa7-9fce-5506a9b491a5 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:54.795601 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:54.804600 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aa4f11-8600-49f8-be9d-894f28bcb9fc None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:54.819715 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4e6696-83dc-4d15-a3ec-eb7cc180e40b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:54.820383 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4e6696-83dc-4d15-a3ec-eb7cc180e40b None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:54.820668 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4e6696-83dc-4d15-a3ec-eb7cc180e40b None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:54.820942 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4e6696-83dc-4d15-a3ec-eb7cc180e40b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:54.830861 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4e6696-83dc-4d15-a3ec-eb7cc180e40b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:54.832337 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4e6696-83dc-4d15-a3ec-eb7cc180e40b None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:54.839440 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df8e570-d936-46d9-b9b0-b9f25313d2cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:54.840048 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df8e570-d936-46d9-b9b0-b9f25313d2cc None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:54.840333 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df8e570-d936-46d9-b9b0-b9f25313d2cc None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:54.840606 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df8e570-d936-46d9-b9b0-b9f25313d2cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:54.848778 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-5c4e6696-83dc-4d15-a3ec-eb7cc180e40b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:21:54.848778 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:54.848778 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:54.848778 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:54.848778 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:54.848778 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:54.848778 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:54.848778 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:54.848778 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:54.848778 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:54.848778 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:54.848778 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:54.848778 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 07:21:54.848778 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 07:21:54.848778 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 07:21:54.848778 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 07:21:54.851401 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:54.851401 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:54.851401 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:54.851401 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:54.851401 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:54.851401 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:54.851401 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:54.851401 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:54.851401 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:54.851401 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:54.851401 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:54.851401 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:54.851401 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:54.851401 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:54.851401 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:54.851401 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:54.851401 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 07:21:54.853519 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 07:21:54.853519 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:21:54.853519 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:21:54.853519 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 539/1099] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:21:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 07:21:54.857109 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4e6696-83dc-4d15-a3ec-eb7cc180e40b None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:54.914475 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df8e570-d936-46d9-b9b0-b9f25313d2cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:54.915934 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df8e570-d936-46d9-b9b0-b9f25313d2cc None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:54.957482 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4e6696-83dc-4d15-a3ec-eb7cc180e40b None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:54.993310 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d76ddee-10a8-4971-8d5a-6a8e047e0908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:54.993955 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d76ddee-10a8-4971-8d5a-6a8e047e0908 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:54.994244 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d76ddee-10a8-4971-8d5a-6a8e047e0908 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:54.994421 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:54.994522 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d76ddee-10a8-4971-8d5a-6a8e047e0908 None admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:54.995151 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d76ddee-10a8-4971-8d5a-6a8e047e0908 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:54.995473 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d76ddee-10a8-4971-8d5a-6a8e047e0908 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:54.996830 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d76ddee-10a8-4971-8d5a-6a8e047e0908 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:55.008474 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 540/1100] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 15 07:21:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:21:55.014146 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d76ddee-10a8-4971-8d5a-6a8e047e0908 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:55.019036 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 561/1101] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 15 07:21:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 15 07:21:55.023428 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df8e570-d936-46d9-b9b0-b9f25313d2cc None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:55.038292 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:55.113775 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d76ddee-10a8-4971-8d5a-6a8e047e0908 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:55.123200 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df8e570-d936-46d9-b9b0-b9f25313d2cc None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:55.147354 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcd71dd-b7cc-4bdd-ba13-b2f6978d4734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:55.147981 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcd71dd-b7cc-4bdd-ba13-b2f6978d4734 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:55.148267 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcd71dd-b7cc-4bdd-ba13-b2f6978d4734 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:55.148544 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcd71dd-b7cc-4bdd-ba13-b2f6978d4734 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:55.155392 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcd71dd-b7cc-4bdd-ba13-b2f6978d4734 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:55.156764 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8943c76-d845-481d-b58e-1a57522856ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:55.156911 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcd71dd-b7cc-4bdd-ba13-b2f6978d4734 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:55.157389 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8943c76-d845-481d-b58e-1a57522856ca None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:55.157705 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8943c76-d845-481d-b58e-1a57522856ca None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:55.157997 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8943c76-d845-481d-b58e-1a57522856ca None admin] PATH_INFO: `/v3/projects/2c7fb9b7cb914824b1a96c70d0653dc0` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:55.159478 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8943c76-d845-481d-b58e-1a57522856ca None admin] RBAC: Authorizing `identity:get_project(project_id=2c7fb9b7cb914824b1a96c70d0653dc0)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:55.160673 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8943c76-d845-481d-b58e-1a57522856ca None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:55.162369 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 562/1102] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 15 07:21:55 2022] GET /identity/v3/projects/2c7fb9b7cb914824b1a96c70d0653dc0 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:21:55.168410 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8943c76-d845-481d-b58e-1a57522856ca None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:55.168569 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-ffcd71dd-b7cc-4bdd-ba13-b2f6978d4734 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 07:21:55.168569 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:21:55.168569 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:21:55.168569 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:21:55.168569 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:21:55.168569 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:21:55.168569 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:21:55.168569 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:21:55.168569 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:21:55.168569 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:21:55.168569 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:21:55.168569 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:21:55.168569 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 07:21:55.168569 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 07:21:55.168569 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 07:21:55.173568 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 07:21:55.173568 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:21:55.173568 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:21:55.173568 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:21:55.173568 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:21:55.173568 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:21:55.173568 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:21:55.173568 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:21:55.173568 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:21:55.173568 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:21:55.173568 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:21:55.173568 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:21:55.173568 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:21:55.173568 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:21:55.173568 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:21:55.173568 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:21:55.173568 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:21:55.175337 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 07:21:55.175337 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 07:21:55.175337 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 07:21:55.175337 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 07:21:55.175337 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 07:21:55.175337 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 07:21:55.175337 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 07:21:55.175337 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:21:55.175337 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 541/1103] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 15 07:21:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 07:21:55.176136 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcd71dd-b7cc-4bdd-ba13-b2f6978d4734 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:55.215196 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:55.257106 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:55.269759 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8943c76-d845-481d-b58e-1a57522856ca None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:55.275073 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcd71dd-b7cc-4bdd-ba13-b2f6978d4734 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:55.304036 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4708f66b-ab15-4930-86dd-8d5ca59d7a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:55.304641 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4708f66b-ab15-4930-86dd-8d5ca59d7a0c None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:55.304922 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4708f66b-ab15-4930-86dd-8d5ca59d7a0c None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:55.305196 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4708f66b-ab15-4930-86dd-8d5ca59d7a0c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:55.305953 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4708f66b-ab15-4930-86dd-8d5ca59d7a0c None admin] RBAC: Adding query filter params (role.id=6bbb643e0d8d4bc0a9b27c26edf59216, scope.project.id=2c7fb9b7cb914824b1a96c70d0653dc0, user.id=fa7775e61bb149d09c4e093f6b19d0c4) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:55.306269 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4708f66b-ab15-4930-86dd-8d5ca59d7a0c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:55.307692 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4708f66b-ab15-4930-86dd-8d5ca59d7a0c None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:55.309973 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7844703d-9886-4881-8287-89e8c2f87919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:55.310582 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7844703d-9886-4881-8287-89e8c2f87919 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:55.310862 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7844703d-9886-4881-8287-89e8c2f87919 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:55.311134 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7844703d-9886-4881-8287-89e8c2f87919 None admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:55.311819 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7844703d-9886-4881-8287-89e8c2f87919 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:55.312124 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7844703d-9886-4881-8287-89e8c2f87919 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:55.313536 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 563/1104] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 15 07:21:55 2022] GET /identity/v3/role_assignments?user.id=fa7775e61bb149d09c4e093f6b19d0c4&scope.project.id=2c7fb9b7cb914824b1a96c70d0653dc0&role.id=6bbb643e0d8d4bc0a9b27c26edf59216 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:21:55.313667 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7844703d-9886-4881-8287-89e8c2f87919 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:55.326237 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 542/1105] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 15 07:21:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:55.333105 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4708f66b-ab15-4930-86dd-8d5ca59d7a0c None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:55.439201 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4708f66b-ab15-4930-86dd-8d5ca59d7a0c None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:55.449900 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:55.476151 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338bca23-4e52-48f8-859f-a75297357200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:55.476771 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338bca23-4e52-48f8-859f-a75297357200 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:55.477051 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338bca23-4e52-48f8-859f-a75297357200 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:55.477999 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338bca23-4e52-48f8-859f-a75297357200 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:55.479010 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338bca23-4e52-48f8-859f-a75297357200 None admin] RBAC: Adding query filter params (role.id=0888721641c54c3684692df3615ea1b0, scope.project.id=4e9987720930473da840579654311bc2, user.id=a080c7ab9b684abca6828067f2572c3b) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:21:55.479557 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338bca23-4e52-48f8-859f-a75297357200 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:55.481340 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338bca23-4e52-48f8-859f-a75297357200 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:55.488111 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 564/1106] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 15 07:21:55 2022] GET /identity/v3/role_assignments?user.id=a080c7ab9b684abca6828067f2572c3b&scope.project.id=4e9987720930473da840579654311bc2&role.id=0888721641c54c3684692df3615ea1b0 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:57.522669 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd579def-867a-4f4b-9bb5-966e8b69a663 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:57.523206 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd579def-867a-4f4b-9bb5-966e8b69a663 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:57.523206 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd579def-867a-4f4b-9bb5-966e8b69a663 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:57.524559 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 543/1107] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:21:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:21:57.530076 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5a169a-1eb2-4f5a-bb73-8f42be53b254 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:57.530510 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5a169a-1eb2-4f5a-bb73-8f42be53b254 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:57.530621 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5a169a-1eb2-4f5a-bb73-8f42be53b254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:57.549379 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-1d5a169a-1eb2-4f5a-bb73-8f42be53b254 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:21:57.550601 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-1d5a169a-1eb2-4f5a-bb73-8f42be53b254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:21:57.614584 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 565/1108] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:21:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 07:21:57.885112 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd579def-867a-4f4b-9bb5-966e8b69a663 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:57.984266 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd579def-867a-4f4b-9bb5-966e8b69a663 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:58.023019 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2ed5aa-7edf-487f-86e9-8b695393a89f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:58.023623 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2ed5aa-7edf-487f-86e9-8b695393a89f None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:58.023899 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2ed5aa-7edf-487f-86e9-8b695393a89f None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:58.024165 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2ed5aa-7edf-487f-86e9-8b695393a89f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:58.025266 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2ed5aa-7edf-487f-86e9-8b695393a89f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:58.026702 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2ed5aa-7edf-487f-86e9-8b695393a89f None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:58.028255 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 544/1109] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:21:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:21:58.034609 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5a169a-1eb2-4f5a-bb73-8f42be53b254 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:21:58.130860 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5a169a-1eb2-4f5a-bb73-8f42be53b254 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:21:58.132320 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:21:58.164385 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773eed5a-6b0e-411c-937f-f89ee62157f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:21:58.164917 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773eed5a-6b0e-411c-937f-f89ee62157f4 None admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:21:58.165193 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773eed5a-6b0e-411c-937f-f89ee62157f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:21:58.165465 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773eed5a-6b0e-411c-937f-f89ee62157f4 None admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:21:58.166129 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773eed5a-6b0e-411c-937f-f89ee62157f4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:21:58.167542 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773eed5a-6b0e-411c-937f-f89ee62157f4 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:21:58.273790 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 566/1110] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Sep 15 07:21:58 2022] POST /identity/v3/users => generated 286 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:22:00.451076 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895de503-48da-4cfe-b13c-1f3b03348e8e None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:00.451605 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895de503-48da-4cfe-b13c-1f3b03348e8e None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:00.451605 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895de503-48da-4cfe-b13c-1f3b03348e8e None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:00.452942 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 545/1111] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:22:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:22:00.458629 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a92557b-940d-4f36-a029-d4e65b459901 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:00.459017 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a92557b-940d-4f36-a029-d4e65b459901 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:00.459184 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a92557b-940d-4f36-a029-d4e65b459901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:00.477462 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-6a92557b-940d-4f36-a029-d4e65b459901 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:22:00.478654 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-6a92557b-940d-4f36-a029-d4e65b459901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:22:00.544503 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 567/1112] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:22:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 07:22:00.825788 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895de503-48da-4cfe-b13c-1f3b03348e8e None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:22:00.838921 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:22:00.924343 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895de503-48da-4cfe-b13c-1f3b03348e8e None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:22:00.958608 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d9fbfe-62b2-4847-93e3-bbbe29b44d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:22:00.959233 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d9fbfe-62b2-4847-93e3-bbbe29b44d47 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:00.959510 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d9fbfe-62b2-4847-93e3-bbbe29b44d47 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:00.959779 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d9fbfe-62b2-4847-93e3-bbbe29b44d47 None admin] PATH_INFO: `/v3/roles/a0fae926490542b4977d1ecf27fcece4` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:00.961146 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d9fbfe-62b2-4847-93e3-bbbe29b44d47 None admin] RBAC: Authorizing `identity:get_role(role_id=a0fae926490542b4977d1ecf27fcece4)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:22:00.962526 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d9fbfe-62b2-4847-93e3-bbbe29b44d47 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:22:00.963845 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 546/1113] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 15 07:22:00 2022] GET /identity/v3/roles/a0fae926490542b4977d1ecf27fcece4 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:22:00.969566 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a92557b-940d-4f36-a029-d4e65b459901 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:22:01.053799 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:22:01.065862 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a92557b-940d-4f36-a029-d4e65b459901 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:22:01.098044 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35d2818-425a-4b05-a06d-6fd966fd3ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:22:01.098604 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35d2818-425a-4b05-a06d-6fd966fd3ba4 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:01.098881 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35d2818-425a-4b05-a06d-6fd966fd3ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:01.099153 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35d2818-425a-4b05-a06d-6fd966fd3ba4 None admin] PATH_INFO: `/v3/users/be73b50406324b1a8eaf2c6c10081ad4` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:01.112513 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35d2818-425a-4b05-a06d-6fd966fd3ba4 None admin] RBAC: Authorizing `identity:get_user(user_id=be73b50406324b1a8eaf2c6c10081ad4)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:22:01.113960 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35d2818-425a-4b05-a06d-6fd966fd3ba4 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:22:01.115525 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 568/1114] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 15 07:22:00 2022] GET /identity/v3/users/be73b50406324b1a8eaf2c6c10081ad4 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:22:01.121265 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d9fbfe-62b2-4847-93e3-bbbe29b44d47 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:22:01.201513 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:22:01.221886 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d9fbfe-62b2-4847-93e3-bbbe29b44d47 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:22:01.327153 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1760a05b-349a-4e39-99b5-2903c49b759b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:22:01.327745 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1760a05b-349a-4e39-99b5-2903c49b759b None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:01.328021 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1760a05b-349a-4e39-99b5-2903c49b759b None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:01.328288 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1760a05b-349a-4e39-99b5-2903c49b759b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:01.393925 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1760a05b-349a-4e39-99b5-2903c49b759b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:22:01.394426 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1760a05b-349a-4e39-99b5-2903c49b759b None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:22:01.467437 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:22:01.489370 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 547/1115] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 15 07:22:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 07:22:01.495388 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35d2818-425a-4b05-a06d-6fd966fd3ba4 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:22:01.592651 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35d2818-425a-4b05-a06d-6fd966fd3ba4 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:22:01.626830 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c717052c-2219-4534-8bf0-4eadc00fd871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:22:01.627456 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c717052c-2219-4534-8bf0-4eadc00fd871 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:01.627831 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c717052c-2219-4534-8bf0-4eadc00fd871 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:01.628109 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c717052c-2219-4534-8bf0-4eadc00fd871 None admin] PATH_INFO: `/v3/projects/2c7fb9b7cb914824b1a96c70d0653dc0` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:01.629220 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c717052c-2219-4534-8bf0-4eadc00fd871 None admin] RBAC: Authorizing `identity:get_project(project_id=2c7fb9b7cb914824b1a96c70d0653dc0)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:22:01.630686 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c717052c-2219-4534-8bf0-4eadc00fd871 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:22:01.632299 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 569/1116] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 15 07:22:01 2022] GET /identity/v3/projects/2c7fb9b7cb914824b1a96c70d0653dc0 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:22:01.638395 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1760a05b-349a-4e39-99b5-2903c49b759b None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:22:01.734515 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1760a05b-349a-4e39-99b5-2903c49b759b None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:22:01.767296 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5ab354-ab89-4e98-9b7c-95c8f43e7846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:22:01.768186 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5ab354-ab89-4e98-9b7c-95c8f43e7846 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:01.768615 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5ab354-ab89-4e98-9b7c-95c8f43e7846 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:01.768899 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5ab354-ab89-4e98-9b7c-95c8f43e7846 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:01.769649 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5ab354-ab89-4e98-9b7c-95c8f43e7846 None admin] RBAC: Adding query filter params (role.id=a0fae926490542b4977d1ecf27fcece4, scope.project.id=2c7fb9b7cb914824b1a96c70d0653dc0, user.id=be73b50406324b1a8eaf2c6c10081ad4) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:22:01.769964 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5ab354-ab89-4e98-9b7c-95c8f43e7846 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:22:01.771374 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5ab354-ab89-4e98-9b7c-95c8f43e7846 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:22:01.776610 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 548/1117] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 15 07:22:01 2022] GET /identity/v3/role_assignments?user.id=be73b50406324b1a8eaf2c6c10081ad4&scope.project.id=2c7fb9b7cb914824b1a96c70d0653dc0&role.id=a0fae926490542b4977d1ecf27fcece4 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:22:03.871348 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63560522-8e39-4797-b6df-9a542d5145c9 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:03.871906 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63560522-8e39-4797-b6df-9a542d5145c9 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:03.871906 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63560522-8e39-4797-b6df-9a542d5145c9 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:03.873238 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 570/1118] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:22:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:22:03.879302 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a711f1-d705-4f18-8884-06bf916aaeb0 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:03.879696 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a711f1-d705-4f18-8884-06bf916aaeb0 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:03.879857 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a711f1-d705-4f18-8884-06bf916aaeb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:03.898579 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-35a711f1-d705-4f18-8884-06bf916aaeb0 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:22:03.899754 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-35a711f1-d705-4f18-8884-06bf916aaeb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:22:03.940339 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:22:03.971494 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 549/1119] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:22:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 07:22:04.242942 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63560522-8e39-4797-b6df-9a542d5145c9 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:22:04.259034 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:22:04.341517 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63560522-8e39-4797-b6df-9a542d5145c9 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:22:04.374199 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edecaa1b-9faa-4368-8d17-52d3d5604a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:22:04.374811 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edecaa1b-9faa-4368-8d17-52d3d5604a8f None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:04.375088 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edecaa1b-9faa-4368-8d17-52d3d5604a8f None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:04.375356 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edecaa1b-9faa-4368-8d17-52d3d5604a8f None admin] PATH_INFO: `/v3/roles/a0fae926490542b4977d1ecf27fcece4` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:04.376742 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edecaa1b-9faa-4368-8d17-52d3d5604a8f None admin] RBAC: Authorizing `identity:get_role(role_id=a0fae926490542b4977d1ecf27fcece4)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:22:04.378922 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edecaa1b-9faa-4368-8d17-52d3d5604a8f None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:22:04.380217 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 571/1120] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 15 07:22:04 2022] GET /identity/v3/roles/a0fae926490542b4977d1ecf27fcece4 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:22:04.386071 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a711f1-d705-4f18-8884-06bf916aaeb0 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:22:04.482070 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a711f1-d705-4f18-8884-06bf916aaeb0 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:22:04.514633 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cee08b-327e-4ddc-bcdd-182e3c4d3c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:22:04.515197 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cee08b-327e-4ddc-bcdd-182e3c4d3c54 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:04.515476 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cee08b-327e-4ddc-bcdd-182e3c4d3c54 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:04.515750 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cee08b-327e-4ddc-bcdd-182e3c4d3c54 None admin] PATH_INFO: `/v3/users/be73b50406324b1a8eaf2c6c10081ad4` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:04.516857 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cee08b-327e-4ddc-bcdd-182e3c4d3c54 None admin] RBAC: Authorizing `identity:get_user(user_id=be73b50406324b1a8eaf2c6c10081ad4)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:22:04.518441 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cee08b-327e-4ddc-bcdd-182e3c4d3c54 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:22:04.520007 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 550/1121] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 15 07:22:04 2022] GET /identity/v3/users/be73b50406324b1a8eaf2c6c10081ad4 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:22:04.525804 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edecaa1b-9faa-4368-8d17-52d3d5604a8f None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:22:04.604072 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:22:04.621833 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edecaa1b-9faa-4368-8d17-52d3d5604a8f None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:22:04.653935 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ab2925-0f98-4d05-b635-e5b903ce4e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:22:04.654547 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ab2925-0f98-4d05-b635-e5b903ce4e52 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:04.654823 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ab2925-0f98-4d05-b635-e5b903ce4e52 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:04.655094 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ab2925-0f98-4d05-b635-e5b903ce4e52 None admin] PATH_INFO: `/v3/projects/2c7fb9b7cb914824b1a96c70d0653dc0` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:04.656186 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ab2925-0f98-4d05-b635-e5b903ce4e52 None admin] RBAC: Authorizing `identity:get_project(project_id=2c7fb9b7cb914824b1a96c70d0653dc0)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:22:04.657626 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ab2925-0f98-4d05-b635-e5b903ce4e52 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:22:04.659825 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 572/1122] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 15 07:22:04 2022] GET /identity/v3/projects/2c7fb9b7cb914824b1a96c70d0653dc0 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:22:04.665725 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cee08b-327e-4ddc-bcdd-182e3c4d3c54 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:22:04.706711 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:22:04.763123 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cee08b-327e-4ddc-bcdd-182e3c4d3c54 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:22:04.796118 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b14ef5-7816-4f0d-b6a4-9a37f07bf43d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:22:04.796764 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b14ef5-7816-4f0d-b6a4-9a37f07bf43d None admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:04.797042 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b14ef5-7816-4f0d-b6a4-9a37f07bf43d None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:04.797312 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b14ef5-7816-4f0d-b6a4-9a37f07bf43d None admin] PATH_INFO: `/v3/projects/2c7fb9b7cb914824b1a96c70d0653dc0/users/be73b50406324b1a8eaf2c6c10081ad4/roles/a0fae926490542b4977d1ecf27fcece4` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:04.800054 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b14ef5-7816-4f0d-b6a4-9a37f07bf43d None admin] RBAC: Authorizing `identity:create_grant(project_id=2c7fb9b7cb914824b1a96c70d0653dc0, user_id=be73b50406324b1a8eaf2c6c10081ad4, role_id=a0fae926490542b4977d1ecf27fcece4)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:22:04.801455 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b14ef5-7816-4f0d-b6a4-9a37f07bf43d None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:22:04.901976 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 551/1123] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Sep 15 07:22:04 2022] PUT /identity/v3/projects/2c7fb9b7cb914824b1a96c70d0653dc0/users/be73b50406324b1a8eaf2c6c10081ad4/roles/a0fae926490542b4977d1ecf27fcece4 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:22:07.079986 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485ee0ea-88fb-4b8d-a4d6-abc814867680 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:07.080501 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485ee0ea-88fb-4b8d-a4d6-abc814867680 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:07.080612 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485ee0ea-88fb-4b8d-a4d6-abc814867680 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:07.081913 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 573/1124] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:22:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:22:07.087147 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae203f00-0f62-4d2f-bd8f-124869a2f2f1 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:07.087564 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae203f00-0f62-4d2f-bd8f-124869a2f2f1 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:07.087672 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae203f00-0f62-4d2f-bd8f-124869a2f2f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:07.105297 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-ae203f00-0f62-4d2f-bd8f-124869a2f2f1 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:22:07.106498 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-ae203f00-0f62-4d2f-bd8f-124869a2f2f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:22:07.169894 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 552/1125] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:22:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 07:22:07.437129 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485ee0ea-88fb-4b8d-a4d6-abc814867680 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:22:07.537026 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485ee0ea-88fb-4b8d-a4d6-abc814867680 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:22:07.569809 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59f58d6-2002-4fc8-90ad-d7d357930bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:22:07.570425 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59f58d6-2002-4fc8-90ad-d7d357930bbf None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:07.570703 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59f58d6-2002-4fc8-90ad-d7d357930bbf None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:07.570976 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59f58d6-2002-4fc8-90ad-d7d357930bbf None admin] PATH_INFO: `/v3/roles/a0fae926490542b4977d1ecf27fcece4` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:07.572327 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59f58d6-2002-4fc8-90ad-d7d357930bbf None admin] RBAC: Authorizing `identity:get_role(role_id=a0fae926490542b4977d1ecf27fcece4)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:22:07.573706 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59f58d6-2002-4fc8-90ad-d7d357930bbf None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:22:07.575007 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 574/1126] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 15 07:22:07 2022] GET /identity/v3/roles/a0fae926490542b4977d1ecf27fcece4 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:22:07.581044 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae203f00-0f62-4d2f-bd8f-124869a2f2f1 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:22:07.610314 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:22:07.683771 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae203f00-0f62-4d2f-bd8f-124869a2f2f1 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:22:07.716510 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa786d9b-e75c-48aa-9628-b1ca356cdaae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:22:07.717074 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa786d9b-e75c-48aa-9628-b1ca356cdaae None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:07.717357 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa786d9b-e75c-48aa-9628-b1ca356cdaae None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:07.717657 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa786d9b-e75c-48aa-9628-b1ca356cdaae None admin] PATH_INFO: `/v3/users/be73b50406324b1a8eaf2c6c10081ad4` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:07.718771 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa786d9b-e75c-48aa-9628-b1ca356cdaae None admin] RBAC: Authorizing `identity:get_user(user_id=be73b50406324b1a8eaf2c6c10081ad4)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:22:07.720187 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa786d9b-e75c-48aa-9628-b1ca356cdaae None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:22:07.722420 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 553/1127] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 15 07:22:07 2022] GET /identity/v3/users/be73b50406324b1a8eaf2c6c10081ad4 => generated 286 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:22:07.729169 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59f58d6-2002-4fc8-90ad-d7d357930bbf None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:22:07.748289 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:22:07.828044 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59f58d6-2002-4fc8-90ad-d7d357930bbf None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:22:07.862288 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409e5717-6c75-456d-94c3-47d23067ae0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:22:07.862882 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409e5717-6c75-456d-94c3-47d23067ae0e None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:07.863162 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409e5717-6c75-456d-94c3-47d23067ae0e None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:07.863434 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409e5717-6c75-456d-94c3-47d23067ae0e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:07.928622 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409e5717-6c75-456d-94c3-47d23067ae0e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:22:07.930111 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409e5717-6c75-456d-94c3-47d23067ae0e None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:22:07.947554 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:22:08.026770 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 575/1128] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 15 07:22:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 07:22:08.032578 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa786d9b-e75c-48aa-9628-b1ca356cdaae None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:22:08.128686 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:22:08.132323 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa786d9b-e75c-48aa-9628-b1ca356cdaae None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:22:08.166457 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4748380e-7480-4bd7-a9f5-02c859e75add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:22:08.167080 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4748380e-7480-4bd7-a9f5-02c859e75add None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:08.167362 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4748380e-7480-4bd7-a9f5-02c859e75add None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:08.167637 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4748380e-7480-4bd7-a9f5-02c859e75add None admin] PATH_INFO: `/v3/projects/2c7fb9b7cb914824b1a96c70d0653dc0` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:08.168740 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4748380e-7480-4bd7-a9f5-02c859e75add None admin] RBAC: Authorizing `identity:get_project(project_id=2c7fb9b7cb914824b1a96c70d0653dc0)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:22:08.170271 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4748380e-7480-4bd7-a9f5-02c859e75add None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:22:08.171897 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 554/1129] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 15 07:22:08 2022] GET /identity/v3/projects/2c7fb9b7cb914824b1a96c70d0653dc0 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:22:08.178473 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409e5717-6c75-456d-94c3-47d23067ae0e None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:22:08.279568 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409e5717-6c75-456d-94c3-47d23067ae0e None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:22:08.296017 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:22:08.312323 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eadbc9-0360-4a9a-884b-247484c1d337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:22:08.312922 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eadbc9-0360-4a9a-884b-247484c1d337 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:08.313202 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eadbc9-0360-4a9a-884b-247484c1d337 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:08.313474 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eadbc9-0360-4a9a-884b-247484c1d337 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:08.314217 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5eadbc9-0360-4a9a-884b-247484c1d337 None admin] RBAC: Adding query filter params (role.id=a0fae926490542b4977d1ecf27fcece4, scope.project.id=2c7fb9b7cb914824b1a96c70d0653dc0, user.id=be73b50406324b1a8eaf2c6c10081ad4) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:22:08.314524 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5eadbc9-0360-4a9a-884b-247484c1d337 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:22:08.316484 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5eadbc9-0360-4a9a-884b-247484c1d337 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:22:08.322270 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 576/1130] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 15 07:22:08 2022] GET /identity/v3/role_assignments?user.id=be73b50406324b1a8eaf2c6c10081ad4&scope.project.id=2c7fb9b7cb914824b1a96c70d0653dc0&role.id=a0fae926490542b4977d1ecf27fcece4 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:22:10.460134 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840451e5-1f98-4d69-8bce-5809926b024d None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:10.460712 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840451e5-1f98-4d69-8bce-5809926b024d None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:10.460712 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840451e5-1f98-4d69-8bce-5809926b024d None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:10.462050 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 555/1131] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:22:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:22:10.467372 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab08f5c7-da7a-4110-8026-2fb5599ad685 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:10.467767 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab08f5c7-da7a-4110-8026-2fb5599ad685 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:10.467922 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab08f5c7-da7a-4110-8026-2fb5599ad685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:10.486308 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-ab08f5c7-da7a-4110-8026-2fb5599ad685 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:22:10.488100 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-ab08f5c7-da7a-4110-8026-2fb5599ad685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:22:10.553346 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 577/1132] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:22:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 07:22:10.822183 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840451e5-1f98-4d69-8bce-5809926b024d None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:22:10.922037 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840451e5-1f98-4d69-8bce-5809926b024d None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:22:10.954776 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c41a594-e425-421a-adbd-cea17bac1b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:22:10.955399 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c41a594-e425-421a-adbd-cea17bac1b4f None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:10.956219 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c41a594-e425-421a-adbd-cea17bac1b4f None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:10.956661 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c41a594-e425-421a-adbd-cea17bac1b4f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:10.958427 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c41a594-e425-421a-adbd-cea17bac1b4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:22:10.959925 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c41a594-e425-421a-adbd-cea17bac1b4f None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:22:10.961541 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 556/1133] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:22:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:22:10.967350 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab08f5c7-da7a-4110-8026-2fb5599ad685 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:22:11.067080 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab08f5c7-da7a-4110-8026-2fb5599ad685 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:22:11.079729 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:22:11.101976 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdbff19-a521-4633-849b-1b522bdf5cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:22:11.102580 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdbff19-a521-4633-849b-1b522bdf5cd7 None admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:11.102860 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdbff19-a521-4633-849b-1b522bdf5cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:11.103136 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdbff19-a521-4633-849b-1b522bdf5cd7 None admin] PATH_INFO: `/v3/projects` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:11.103787 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdbff19-a521-4633-849b-1b522bdf5cd7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:22:11.105206 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdbff19-a521-4633-849b-1b522bdf5cd7 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:22:11.166279 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 578/1134] 10.222.0.42 () {60 vars in 1141 bytes} [Thu Sep 15 07:22:10 2022] POST /identity/v3/projects => generated 311 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:22:13.384267 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d093b7-4153-4b5e-81de-f58257b6c90d None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:13.384797 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d093b7-4153-4b5e-81de-f58257b6c90d None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:13.384797 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d093b7-4153-4b5e-81de-f58257b6c90d None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:13.386186 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 557/1135] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:22:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:22:13.391191 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b037d0f7-6e08-4f22-82c8-575426870818 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:13.391558 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b037d0f7-6e08-4f22-82c8-575426870818 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:13.391731 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b037d0f7-6e08-4f22-82c8-575426870818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:13.410344 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-b037d0f7-6e08-4f22-82c8-575426870818 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:22:13.411501 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-b037d0f7-6e08-4f22-82c8-575426870818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:22:13.477211 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 579/1136] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:22:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 07:22:13.745931 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d093b7-4153-4b5e-81de-f58257b6c90d None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:22:13.779862 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:22:13.846136 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d093b7-4153-4b5e-81de-f58257b6c90d None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:22:13.879021 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee32218-7cad-42d8-87e1-dd193fcc2f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:22:13.879644 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee32218-7cad-42d8-87e1-dd193fcc2f80 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:13.879928 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee32218-7cad-42d8-87e1-dd193fcc2f80 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:13.880205 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee32218-7cad-42d8-87e1-dd193fcc2f80 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:13.881309 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee32218-7cad-42d8-87e1-dd193fcc2f80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:22:13.882764 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee32218-7cad-42d8-87e1-dd193fcc2f80 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:22:13.884324 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 558/1137] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:22:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:22:13.890089 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b037d0f7-6e08-4f22-82c8-575426870818 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:22:13.981214 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:22:13.987760 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b037d0f7-6e08-4f22-82c8-575426870818 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:22:14.021363 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a89874-19ba-445a-b918-122c1c8e6407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:22:14.021925 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a89874-19ba-445a-b918-122c1c8e6407 None admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:14.022209 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a89874-19ba-445a-b918-122c1c8e6407 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:14.022484 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a89874-19ba-445a-b918-122c1c8e6407 None admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:14.023129 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a89874-19ba-445a-b918-122c1c8e6407 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:22:14.024554 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a89874-19ba-445a-b918-122c1c8e6407 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:22:14.114485 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 580/1138] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Sep 15 07:22:13 2022] POST /identity/v3/users => generated 286 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:22:16.287454 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cce0fa4-925a-4535-ac37-bd3ccc06bc62 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:16.287970 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cce0fa4-925a-4535-ac37-bd3ccc06bc62 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:16.287970 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cce0fa4-925a-4535-ac37-bd3ccc06bc62 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:16.289260 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 559/1139] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:22:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:22:16.294371 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb3a7e4-a7e5-4c01-9fbb-dab554a64db0 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:16.294813 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb3a7e4-a7e5-4c01-9fbb-dab554a64db0 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:16.294920 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb3a7e4-a7e5-4c01-9fbb-dab554a64db0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:16.313217 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-5bb3a7e4-a7e5-4c01-9fbb-dab554a64db0 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:22:16.314419 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-5bb3a7e4-a7e5-4c01-9fbb-dab554a64db0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:22:16.378941 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 581/1140] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:22:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 07:22:16.651888 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cce0fa4-925a-4535-ac37-bd3ccc06bc62 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:22:16.751848 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cce0fa4-925a-4535-ac37-bd3ccc06bc62 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:22:16.753421 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:22:16.785150 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218a6e24-d3f9-4800-91cb-fe1b77cfd1a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:22:16.785833 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218a6e24-d3f9-4800-91cb-fe1b77cfd1a1 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:16.786115 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218a6e24-d3f9-4800-91cb-fe1b77cfd1a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:16.786394 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218a6e24-d3f9-4800-91cb-fe1b77cfd1a1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:16.796648 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218a6e24-d3f9-4800-91cb-fe1b77cfd1a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:22:16.802053 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-218a6e24-d3f9-4800-91cb-fe1b77cfd1a1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 07:22:16.802053 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:22:16.802053 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:22:16.802053 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:22:16.802053 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:22:16.802053 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:22:16.802053 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:22:16.802053 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:22:16.802053 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:22:16.802053 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:22:16.802053 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:22:16.802053 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:22:16.802053 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:22:16.802053 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:22:16.802053 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:22:16.802053 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:22:16.803169 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:22:16.803169 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:22:16.803169 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:22:16.803169 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:22:16.803169 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:22:16.803169 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:22:16.803169 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:22:16.803169 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:22:16.803169 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:22:16.803169 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:22:16.803169 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:22:16.803169 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:22:16.803169 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:22:16.803169 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:22:16.803169 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:22:16.803169 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:22:16.804029 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:22:16.804029 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:22:16.804029 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:22:16.804029 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:22:16.804029 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:22:16.804029 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:22:16.804029 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:22:16.804029 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:22:16.804029 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:22:16.804029 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:22:16.804029 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 07:22:16.804029 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:22:16.804029 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 560/1141] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 15 07:22:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:22:16.811184 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb3a7e4-a7e5-4c01-9fbb-dab554a64db0 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:22:16.889390 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:22:16.908401 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb3a7e4-a7e5-4c01-9fbb-dab554a64db0 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:22:16.942236 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a2cd4d-cdb4-4e93-a943-20f2c752fe90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:22:16.942844 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a2cd4d-cdb4-4e93-a943-20f2c752fe90 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:16.943125 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a2cd4d-cdb4-4e93-a943-20f2c752fe90 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:16.943726 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a2cd4d-cdb4-4e93-a943-20f2c752fe90 None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:16.944052 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a2cd4d-cdb4-4e93-a943-20f2c752fe90 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:22:16.944358 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a2cd4d-cdb4-4e93-a943-20f2c752fe90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:22:16.945775 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a2cd4d-cdb4-4e93-a943-20f2c752fe90 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:22:16.954734 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 582/1142] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 15 07:22:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:22:16.961394 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218a6e24-d3f9-4800-91cb-fe1b77cfd1a1 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:22:17.059875 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218a6e24-d3f9-4800-91cb-fe1b77cfd1a1 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:22:17.093119 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90933e17-2733-4979-8775-4c60f4ae252a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:22:17.093764 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90933e17-2733-4979-8775-4c60f4ae252a None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:17.094045 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90933e17-2733-4979-8775-4c60f4ae252a None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:17.094318 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90933e17-2733-4979-8775-4c60f4ae252a None admin] PATH_INFO: `/v3/users/9714a0cb21ef4c1eacc4c10c6ef7bc25` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:17.107142 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90933e17-2733-4979-8775-4c60f4ae252a None admin] RBAC: Authorizing `identity:get_user(user_id=9714a0cb21ef4c1eacc4c10c6ef7bc25)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:22:17.108572 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90933e17-2733-4979-8775-4c60f4ae252a None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:22:17.110199 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 561/1143] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 15 07:22:16 2022] GET /identity/v3/users/9714a0cb21ef4c1eacc4c10c6ef7bc25 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:22:17.116593 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a2cd4d-cdb4-4e93-a943-20f2c752fe90 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:22:17.215338 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a2cd4d-cdb4-4e93-a943-20f2c752fe90 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:22:17.248796 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c5b514-61fb-4f15-924c-f73d7a9f7e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:22:17.249390 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c5b514-61fb-4f15-924c-f73d7a9f7e7b None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:17.249701 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c5b514-61fb-4f15-924c-f73d7a9f7e7b None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:17.249976 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c5b514-61fb-4f15-924c-f73d7a9f7e7b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:17.261697 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:22:17.316629 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c5b514-61fb-4f15-924c-f73d7a9f7e7b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:22:17.318099 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c5b514-61fb-4f15-924c-f73d7a9f7e7b None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:22:17.414608 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 583/1144] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 15 07:22:17 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 07:22:17.420634 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90933e17-2733-4979-8775-4c60f4ae252a None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:22:17.443102 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:22:17.519267 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90933e17-2733-4979-8775-4c60f4ae252a None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:22:17.551983 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aef328d-3c50-47fd-a6f7-9bcd2dd27339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:22:17.552605 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aef328d-3c50-47fd-a6f7-9bcd2dd27339 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:17.552887 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aef328d-3c50-47fd-a6f7-9bcd2dd27339 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:17.553160 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aef328d-3c50-47fd-a6f7-9bcd2dd27339 None admin] PATH_INFO: `/v3/projects/c93e2330dbf446bd95394f07fe34fed6` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:17.554301 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aef328d-3c50-47fd-a6f7-9bcd2dd27339 None admin] RBAC: Authorizing `identity:get_project(project_id=c93e2330dbf446bd95394f07fe34fed6)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:22:17.555715 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aef328d-3c50-47fd-a6f7-9bcd2dd27339 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:22:17.557332 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 562/1145] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 15 07:22:17 2022] GET /identity/v3/projects/c93e2330dbf446bd95394f07fe34fed6 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:22:17.563354 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c5b514-61fb-4f15-924c-f73d7a9f7e7b None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:22:17.606628 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:22:17.662463 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c5b514-61fb-4f15-924c-f73d7a9f7e7b None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:22:17.695275 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d807d8-047f-4eb3-acb5-7da9ed65bdcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:22:17.695869 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d807d8-047f-4eb3-acb5-7da9ed65bdcb None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:17.696150 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d807d8-047f-4eb3-acb5-7da9ed65bdcb None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:17.696426 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d807d8-047f-4eb3-acb5-7da9ed65bdcb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:17.697142 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d807d8-047f-4eb3-acb5-7da9ed65bdcb None admin] RBAC: Adding query filter params (role.id=6bbb643e0d8d4bc0a9b27c26edf59216, scope.project.id=c93e2330dbf446bd95394f07fe34fed6, user.id=9714a0cb21ef4c1eacc4c10c6ef7bc25) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:22:17.697451 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d807d8-047f-4eb3-acb5-7da9ed65bdcb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:22:17.698903 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d807d8-047f-4eb3-acb5-7da9ed65bdcb None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:22:17.704078 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 584/1146] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 15 07:22:17 2022] GET /identity/v3/role_assignments?user.id=9714a0cb21ef4c1eacc4c10c6ef7bc25&scope.project.id=c93e2330dbf446bd95394f07fe34fed6&role.id=6bbb643e0d8d4bc0a9b27c26edf59216 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:22:19.883789 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3a02c3-397c-4bbc-820b-66dcd3a6f662 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:19.884312 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3a02c3-397c-4bbc-820b-66dcd3a6f662 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:19.884312 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3a02c3-397c-4bbc-820b-66dcd3a6f662 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:19.885699 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 563/1147] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:22:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:22:19.890971 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404036aa-2a07-4638-89d7-cd200e3efdb0 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:19.891389 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404036aa-2a07-4638-89d7-cd200e3efdb0 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:19.891520 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404036aa-2a07-4638-89d7-cd200e3efdb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:19.910380 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-404036aa-2a07-4638-89d7-cd200e3efdb0 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:22:19.911537 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-404036aa-2a07-4638-89d7-cd200e3efdb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:22:19.977492 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 585/1148] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:22:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 07:22:20.243673 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3a02c3-397c-4bbc-820b-66dcd3a6f662 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:22:20.339111 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:22:20.351457 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3a02c3-397c-4bbc-820b-66dcd3a6f662 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:22:20.384761 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6b63bd-3be0-4617-8c25-4f19c80adbb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:22:20.385383 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b63bd-3be0-4617-8c25-4f19c80adbb1 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:20.385692 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b63bd-3be0-4617-8c25-4f19c80adbb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:20.385973 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b63bd-3be0-4617-8c25-4f19c80adbb1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:20.396384 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6b63bd-3be0-4617-8c25-4f19c80adbb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:22:20.401169 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-7d6b63bd-3be0-4617-8c25-4f19c80adbb1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 07:22:20.401169 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:22:20.401169 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:22:20.401169 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:22:20.401169 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:22:20.401169 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:22:20.401169 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:22:20.401169 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:22:20.401169 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:22:20.401169 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:22:20.401169 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:22:20.401169 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:22:20.401169 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:22:20.401169 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:22:20.401169 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:22:20.401169 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:22:20.402922 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:22:20.402922 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:22:20.402922 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:22:20.402922 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:22:20.402922 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:22:20.402922 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:22:20.402922 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:22:20.402922 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:22:20.402922 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:22:20.402922 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:22:20.402922 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:22:20.402922 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:22:20.402922 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:22:20.402922 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:22:20.402922 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:22:20.402922 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:22:20.404586 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:22:20.404586 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:22:20.404586 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:22:20.404586 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:22:20.404586 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:22:20.404586 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:22:20.404586 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:22:20.404586 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:22:20.404586 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:22:20.404586 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:22:20.404586 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 07:22:20.404586 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:22:20.404586 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 564/1149] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 15 07:22:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:22:20.408403 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404036aa-2a07-4638-89d7-cd200e3efdb0 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:22:20.442050 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:22:20.504892 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404036aa-2a07-4638-89d7-cd200e3efdb0 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:22:20.538303 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6699fe1-9c75-4ae2-bdfc-2919250fabaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:22:20.538899 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6699fe1-9c75-4ae2-bdfc-2919250fabaa None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:20.539176 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6699fe1-9c75-4ae2-bdfc-2919250fabaa None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:20.539447 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6699fe1-9c75-4ae2-bdfc-2919250fabaa None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:20.540123 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6699fe1-9c75-4ae2-bdfc-2919250fabaa None admin] RBAC: Adding query filter params (name=admin) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:22:20.540426 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6699fe1-9c75-4ae2-bdfc-2919250fabaa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:22:20.541806 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6699fe1-9c75-4ae2-bdfc-2919250fabaa None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:22:20.550390 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 586/1150] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 15 07:22:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:22:20.556435 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6b63bd-3be0-4617-8c25-4f19c80adbb1 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:22:20.655199 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6b63bd-3be0-4617-8c25-4f19c80adbb1 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:22:20.687459 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368cf3ed-770f-4d30-bd4d-23113a6d3b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:22:20.688077 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368cf3ed-770f-4d30-bd4d-23113a6d3b49 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:20.688359 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368cf3ed-770f-4d30-bd4d-23113a6d3b49 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:20.688629 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368cf3ed-770f-4d30-bd4d-23113a6d3b49 None admin] PATH_INFO: `/v3/users/9714a0cb21ef4c1eacc4c10c6ef7bc25` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:20.689762 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368cf3ed-770f-4d30-bd4d-23113a6d3b49 None admin] RBAC: Authorizing `identity:get_user(user_id=9714a0cb21ef4c1eacc4c10c6ef7bc25)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:22:20.691972 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368cf3ed-770f-4d30-bd4d-23113a6d3b49 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:22:20.693524 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 565/1151] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 15 07:22:20 2022] GET /identity/v3/users/9714a0cb21ef4c1eacc4c10c6ef7bc25 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:22:20.699236 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6699fe1-9c75-4ae2-bdfc-2919250fabaa None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:22:20.794534 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:22:20.798435 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6699fe1-9c75-4ae2-bdfc-2919250fabaa None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:22:20.832085 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2dd674-dcad-4206-96d9-aed9cdb4641d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:22:20.832718 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2dd674-dcad-4206-96d9-aed9cdb4641d None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:20.833004 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2dd674-dcad-4206-96d9-aed9cdb4641d None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:20.833284 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2dd674-dcad-4206-96d9-aed9cdb4641d None admin] PATH_INFO: `/v3/projects/c93e2330dbf446bd95394f07fe34fed6` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:20.834431 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2dd674-dcad-4206-96d9-aed9cdb4641d None admin] RBAC: Authorizing `identity:get_project(project_id=c93e2330dbf446bd95394f07fe34fed6)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:22:20.835863 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2dd674-dcad-4206-96d9-aed9cdb4641d None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:22:20.837472 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 587/1152] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 15 07:22:20 2022] GET /identity/v3/projects/c93e2330dbf446bd95394f07fe34fed6 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:22:20.843323 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368cf3ed-770f-4d30-bd4d-23113a6d3b49 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:22:20.862070 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:22:20.942338 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368cf3ed-770f-4d30-bd4d-23113a6d3b49 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:22:20.975858 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96eb307-c2c6-4bbd-88ff-fac18bc20fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:22:20.976515 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96eb307-c2c6-4bbd-88ff-fac18bc20fbe None admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:20.976795 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96eb307-c2c6-4bbd-88ff-fac18bc20fbe None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:20.977069 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96eb307-c2c6-4bbd-88ff-fac18bc20fbe None admin] PATH_INFO: `/v3/projects/c93e2330dbf446bd95394f07fe34fed6/users/9714a0cb21ef4c1eacc4c10c6ef7bc25/roles/6bbb643e0d8d4bc0a9b27c26edf59216` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:20.978980 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96eb307-c2c6-4bbd-88ff-fac18bc20fbe None admin] RBAC: Authorizing `identity:create_grant(project_id=c93e2330dbf446bd95394f07fe34fed6, user_id=9714a0cb21ef4c1eacc4c10c6ef7bc25, role_id=6bbb643e0d8d4bc0a9b27c26edf59216)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:22:20.980396 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96eb307-c2c6-4bbd-88ff-fac18bc20fbe None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:22:21.049616 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 566/1153] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Sep 15 07:22:20 2022] PUT /identity/v3/projects/c93e2330dbf446bd95394f07fe34fed6/users/9714a0cb21ef4c1eacc4c10c6ef7bc25/roles/6bbb643e0d8d4bc0a9b27c26edf59216 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:22:23.212168 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadb6e4e-b01a-49b7-be57-4a2ed5916934 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:23.212670 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadb6e4e-b01a-49b7-be57-4a2ed5916934 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:23.212670 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadb6e4e-b01a-49b7-be57-4a2ed5916934 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:23.214093 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 588/1154] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:22:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:22:23.219409 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534cdc2f-616d-421f-95ea-e72beeaf819b None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:23.219834 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534cdc2f-616d-421f-95ea-e72beeaf819b None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:23.219964 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534cdc2f-616d-421f-95ea-e72beeaf819b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:23.237799 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-534cdc2f-616d-421f-95ea-e72beeaf819b None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:22:23.239017 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-534cdc2f-616d-421f-95ea-e72beeaf819b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:22:23.303167 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 567/1155] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:22:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 07:22:23.568734 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadb6e4e-b01a-49b7-be57-4a2ed5916934 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:22:23.668984 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadb6e4e-b01a-49b7-be57-4a2ed5916934 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:22:23.701680 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e391746-cb97-4028-bb06-2a1a7da1216c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:22:23.702299 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e391746-cb97-4028-bb06-2a1a7da1216c None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:23.702582 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e391746-cb97-4028-bb06-2a1a7da1216c None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:23.702855 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e391746-cb97-4028-bb06-2a1a7da1216c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:23.713882 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e391746-cb97-4028-bb06-2a1a7da1216c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:22:23.718628 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-5e391746-cb97-4028-bb06-2a1a7da1216c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 07:22:23.718628 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:22:23.718628 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:22:23.718628 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:22:23.718628 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:22:23.718628 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:22:23.718628 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:22:23.718628 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:22:23.718628 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:22:23.718628 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:22:23.718628 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:22:23.718628 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:22:23.718628 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:22:23.718628 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:22:23.718628 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:22:23.718628 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:22:23.719791 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:22:23.719791 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:22:23.719791 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:22:23.719791 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:22:23.719791 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:22:23.719791 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:22:23.719791 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:22:23.719791 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:22:23.719791 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:22:23.719791 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:22:23.719791 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:22:23.719791 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:22:23.719791 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:22:23.719791 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:22:23.719791 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:22:23.719791 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:22:23.720853 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:22:23.720853 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:22:23.720853 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:22:23.720853 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:22:23.720853 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:22:23.720853 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:22:23.720853 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:22:23.720853 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:22:23.720853 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:22:23.720853 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:22:23.720853 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 07:22:23.720853 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:22:23.720853 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 589/1156] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 15 07:22:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:22:23.725717 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534cdc2f-616d-421f-95ea-e72beeaf819b None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:22:23.733758 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:22:23.823103 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534cdc2f-616d-421f-95ea-e72beeaf819b None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:22:23.855995 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d558c18-b3dd-4fba-b892-c84b4b4d1249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:22:23.856572 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d558c18-b3dd-4fba-b892-c84b4b4d1249 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:23.856850 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d558c18-b3dd-4fba-b892-c84b4b4d1249 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:23.857118 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d558c18-b3dd-4fba-b892-c84b4b4d1249 None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:23.857793 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d558c18-b3dd-4fba-b892-c84b4b4d1249 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:22:23.858101 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d558c18-b3dd-4fba-b892-c84b4b4d1249 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:22:23.859470 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d558c18-b3dd-4fba-b892-c84b4b4d1249 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:22:23.867884 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 568/1157] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 15 07:22:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:22:23.873750 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e391746-cb97-4028-bb06-2a1a7da1216c None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:22:23.901315 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:22:23.975063 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e391746-cb97-4028-bb06-2a1a7da1216c None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:22:24.011977 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaf5d9f-c1c5-4198-ba00-8e7dcf3eacef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:22:24.012604 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaf5d9f-c1c5-4198-ba00-8e7dcf3eacef None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:24.012887 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaf5d9f-c1c5-4198-ba00-8e7dcf3eacef None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:24.013164 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaf5d9f-c1c5-4198-ba00-8e7dcf3eacef None admin] PATH_INFO: `/v3/users/9714a0cb21ef4c1eacc4c10c6ef7bc25` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:24.014315 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaf5d9f-c1c5-4198-ba00-8e7dcf3eacef None admin] RBAC: Authorizing `identity:get_user(user_id=9714a0cb21ef4c1eacc4c10c6ef7bc25)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:22:24.015752 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaf5d9f-c1c5-4198-ba00-8e7dcf3eacef None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:22:24.017323 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 590/1158] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 15 07:22:23 2022] GET /identity/v3/users/9714a0cb21ef4c1eacc4c10c6ef7bc25 => generated 286 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:22:24.023399 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d558c18-b3dd-4fba-b892-c84b4b4d1249 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:22:24.088210 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:22:24.121761 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d558c18-b3dd-4fba-b892-c84b4b4d1249 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:22:24.154102 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b3bd44-00c8-40ff-a92d-fa24636dabfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:22:24.154706 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b3bd44-00c8-40ff-a92d-fa24636dabfb None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:24.154991 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b3bd44-00c8-40ff-a92d-fa24636dabfb None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:24.155267 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b3bd44-00c8-40ff-a92d-fa24636dabfb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:24.220984 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b3bd44-00c8-40ff-a92d-fa24636dabfb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:22:24.223036 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b3bd44-00c8-40ff-a92d-fa24636dabfb None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:22:24.291628 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:22:24.326176 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 569/1159] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 15 07:22:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 07:22:24.332211 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaf5d9f-c1c5-4198-ba00-8e7dcf3eacef None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:22:24.423411 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:22:24.430729 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaf5d9f-c1c5-4198-ba00-8e7dcf3eacef None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:22:24.468462 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794ea315-279a-4174-9eb3-512243ef6823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:22:24.469108 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794ea315-279a-4174-9eb3-512243ef6823 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:24.469391 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794ea315-279a-4174-9eb3-512243ef6823 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:24.469692 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794ea315-279a-4174-9eb3-512243ef6823 None admin] PATH_INFO: `/v3/projects/c93e2330dbf446bd95394f07fe34fed6` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:24.470815 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794ea315-279a-4174-9eb3-512243ef6823 None admin] RBAC: Authorizing `identity:get_project(project_id=c93e2330dbf446bd95394f07fe34fed6)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:22:24.472250 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794ea315-279a-4174-9eb3-512243ef6823 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:22:24.479023 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 591/1160] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 15 07:22:24 2022] GET /identity/v3/projects/c93e2330dbf446bd95394f07fe34fed6 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:22:24.480209 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b3bd44-00c8-40ff-a92d-fa24636dabfb None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:22:24.580447 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b3bd44-00c8-40ff-a92d-fa24636dabfb None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:22:24.615200 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2413bf65-03f8-4f2b-9f3e-9a505a3a1735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:22:24.615809 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2413bf65-03f8-4f2b-9f3e-9a505a3a1735 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:24.616092 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2413bf65-03f8-4f2b-9f3e-9a505a3a1735 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:24.616367 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2413bf65-03f8-4f2b-9f3e-9a505a3a1735 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:24.617092 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2413bf65-03f8-4f2b-9f3e-9a505a3a1735 None admin] RBAC: Adding query filter params (role.id=6bbb643e0d8d4bc0a9b27c26edf59216, scope.project.id=c93e2330dbf446bd95394f07fe34fed6, user.id=9714a0cb21ef4c1eacc4c10c6ef7bc25) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:22:24.617404 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2413bf65-03f8-4f2b-9f3e-9a505a3a1735 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:22:24.619356 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2413bf65-03f8-4f2b-9f3e-9a505a3a1735 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:22:24.630979 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 570/1161] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 15 07:22:24 2022] GET /identity/v3/role_assignments?user.id=9714a0cb21ef4c1eacc4c10c6ef7bc25&scope.project.id=c93e2330dbf446bd95394f07fe34fed6&role.id=6bbb643e0d8d4bc0a9b27c26edf59216 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:22:26.804589 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5fee73-d5b4-4b28-9c8f-0e1f564dca00 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:26.805142 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5fee73-d5b4-4b28-9c8f-0e1f564dca00 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:26.805142 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5fee73-d5b4-4b28-9c8f-0e1f564dca00 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:26.806492 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 592/1162] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:22:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:22:26.811797 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d8f427-fdc5-4c0a-8663-476c6f5e29ed None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:26.812208 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d8f427-fdc5-4c0a-8663-476c6f5e29ed None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:26.812351 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d8f427-fdc5-4c0a-8663-476c6f5e29ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:26.830893 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-f9d8f427-fdc5-4c0a-8663-476c6f5e29ed None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:22:26.832107 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-f9d8f427-fdc5-4c0a-8663-476c6f5e29ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:22:26.856535 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:22:26.897157 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 571/1163] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:22:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 07:22:27.174038 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5fee73-d5b4-4b28-9c8f-0e1f564dca00 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:22:27.275916 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5fee73-d5b4-4b28-9c8f-0e1f564dca00 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:22:27.309053 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6061e077-1128-4b42-b073-177186556076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:22:27.309683 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6061e077-1128-4b42-b073-177186556076 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:27.309968 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6061e077-1128-4b42-b073-177186556076 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:27.310244 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6061e077-1128-4b42-b073-177186556076 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:27.339489 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:22:27.375965 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6061e077-1128-4b42-b073-177186556076 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:22:27.377453 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6061e077-1128-4b42-b073-177186556076 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:22:27.490784 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 593/1164] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 15 07:22:27 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 07:22:27.496613 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d8f427-fdc5-4c0a-8663-476c6f5e29ed None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:22:27.591903 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d8f427-fdc5-4c0a-8663-476c6f5e29ed None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:22:27.624616 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dffdb8a-51ff-4ee2-9cb6-79af77a100df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:22:27.625231 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dffdb8a-51ff-4ee2-9cb6-79af77a100df None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:27.625507 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dffdb8a-51ff-4ee2-9cb6-79af77a100df None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:27.625809 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dffdb8a-51ff-4ee2-9cb6-79af77a100df None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:27.632250 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dffdb8a-51ff-4ee2-9cb6-79af77a100df None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:22:27.633692 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dffdb8a-51ff-4ee2-9cb6-79af77a100df None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:22:27.645376 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-6dffdb8a-51ff-4ee2-9cb6-79af77a100df None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 15 07:22:27.645376 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:22:27.645376 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 15 07:22:27.645376 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 15 07:22:27.645376 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 07:22:27.645376 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 07:22:27.645376 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 07:22:27.645376 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 07:22:27.645376 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 15 07:22:27.645376 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:22:27.645376 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 07:22:27.645376 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:22:27.645376 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:22:27.645376 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:22:27.645376 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:22:27.645376 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:22:27.645376 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:22:27.646684 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:22:27.646684 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:22:27.646684 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:22:27.646684 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:22:27.646684 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:22:27.646684 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:22:27.646684 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 07:22:27.646684 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 07:22:27.646684 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 07:22:27.646684 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 07:22:27.646684 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:22:27.646684 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:22:27.646684 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:22:27.646684 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:22:27.646684 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:22:27.646684 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:22:27.647738 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:22:27.647738 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:22:27.647738 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:22:27.647738 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:22:27.647738 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:22:27.647738 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:22:27.647738 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:22:27.647738 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:22:27.647738 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:22:27.647738 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:22:27.647738 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 15 07:22:27.647738 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 07:22:27.647738 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 15 07:22:27.647738 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:22:27.647738 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 572/1165] 10.222.0.42 () {56 vars in 1117 bytes} [Thu Sep 15 07:22:27 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:22:27.652368 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6061e077-1128-4b42-b073-177186556076 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:22:27.719794 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:22:27.761116 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6061e077-1128-4b42-b073-177186556076 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:22:27.794867 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec90370-7d95-4ed8-8668-db0c2c6892ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:22:27.795475 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec90370-7d95-4ed8-8668-db0c2c6892ca None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:27.795758 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec90370-7d95-4ed8-8668-db0c2c6892ca None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:27.796034 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec90370-7d95-4ed8-8668-db0c2c6892ca None admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:27.796667 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec90370-7d95-4ed8-8668-db0c2c6892ca None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:22:27.797102 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec90370-7d95-4ed8-8668-db0c2c6892ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:22:27.798505 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec90370-7d95-4ed8-8668-db0c2c6892ca None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:22:27.806473 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 594/1166] 10.222.0.42 () {56 vars in 1115 bytes} [Thu Sep 15 07:22:27 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:22:29.948102 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6484437a-d768-425c-a2b4-8cd9b4567f12 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:29.948501 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6484437a-d768-425c-a2b4-8cd9b4567f12 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:29.948645 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6484437a-d768-425c-a2b4-8cd9b4567f12 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:29.950043 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 573/1167] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:22:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:22:29.955459 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5db5b6-25fe-4dec-a293-05cdd1aad98d None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:29.955854 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5db5b6-25fe-4dec-a293-05cdd1aad98d None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:29.956010 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5db5b6-25fe-4dec-a293-05cdd1aad98d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:29.974536 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-5c5db5b6-25fe-4dec-a293-05cdd1aad98d None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:22:29.975696 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-5c5db5b6-25fe-4dec-a293-05cdd1aad98d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:22:30.040689 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 595/1168] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:22:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 07:22:30.306268 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6484437a-d768-425c-a2b4-8cd9b4567f12 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:22:30.332253 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:22:30.408517 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6484437a-d768-425c-a2b4-8cd9b4567f12 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:22:30.441229 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854ceb29-c5d9-4706-b294-4ecf61073d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:22:30.442196 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854ceb29-c5d9-4706-b294-4ecf61073d51 None admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:30.442470 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854ceb29-c5d9-4706-b294-4ecf61073d51 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:30.442740 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854ceb29-c5d9-4706-b294-4ecf61073d51 None admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:30.443410 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854ceb29-c5d9-4706-b294-4ecf61073d51 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:22:30.444758 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854ceb29-c5d9-4706-b294-4ecf61073d51 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:22:30.544473 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 574/1169] 10.222.0.42 () {60 vars in 1138 bytes} [Thu Sep 15 07:22:30 2022] POST /identity/v3/domains => generated 267 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:22:32.702575 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc18235b-40bd-4efa-b310-a62d9a951b9c None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:32.703120 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc18235b-40bd-4efa-b310-a62d9a951b9c None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:32.703120 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc18235b-40bd-4efa-b310-a62d9a951b9c None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:32.704448 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 596/1170] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:22:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:22:32.710462 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ba26e6-4157-428b-86ad-1dc448b6928d None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:32.710855 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ba26e6-4157-428b-86ad-1dc448b6928d None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:32.711008 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ba26e6-4157-428b-86ad-1dc448b6928d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:32.729385 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-b0ba26e6-4157-428b-86ad-1dc448b6928d None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:22:32.730578 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-b0ba26e6-4157-428b-86ad-1dc448b6928d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:22:32.795156 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 575/1171] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:22:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 07:22:33.060409 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc18235b-40bd-4efa-b310-a62d9a951b9c None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:22:33.117065 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:22:33.159866 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc18235b-40bd-4efa-b310-a62d9a951b9c None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:22:33.193997 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b40f050-af9e-4f99-9795-a9bb5d17295a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:22:33.194611 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b40f050-af9e-4f99-9795-a9bb5d17295a None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:33.194888 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b40f050-af9e-4f99-9795-a9bb5d17295a None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:33.195160 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b40f050-af9e-4f99-9795-a9bb5d17295a None admin] PATH_INFO: `/v3/domains/b44c9c7b1b624302ab672b506886a69b` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:33.205196 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b40f050-af9e-4f99-9795-a9bb5d17295a None admin] RBAC: Authorizing `identity:get_domain(domain_id=b44c9c7b1b624302ab672b506886a69b)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:22:33.206655 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b40f050-af9e-4f99-9795-a9bb5d17295a None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:22:33.208888 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 597/1172] 10.222.0.42 () {56 vars in 1183 bytes} [Thu Sep 15 07:22:33 2022] GET /identity/v3/domains/b44c9c7b1b624302ab672b506886a69b => generated 267 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:22:33.214722 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ba26e6-4157-428b-86ad-1dc448b6928d None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:22:33.228666 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:22:33.310483 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ba26e6-4157-428b-86ad-1dc448b6928d None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:22:33.343772 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971a1ab6-03da-4685-af7b-c438f5f316c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:22:33.344365 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971a1ab6-03da-4685-af7b-c438f5f316c1 None admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:33.344642 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971a1ab6-03da-4685-af7b-c438f5f316c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:33.344911 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971a1ab6-03da-4685-af7b-c438f5f316c1 None admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:33.345558 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971a1ab6-03da-4685-af7b-c438f5f316c1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:22:33.346984 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971a1ab6-03da-4685-af7b-c438f5f316c1 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:22:33.434183 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 576/1173] 10.222.0.42 () {60 vars in 1141 bytes} [Thu Sep 15 07:22:33 2022] POST /identity/v3/projects => generated 361 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:22:35.585984 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cb58b1-d209-4102-93b5-57a5fe8bfb94 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:35.586543 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cb58b1-d209-4102-93b5-57a5fe8bfb94 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:35.586543 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cb58b1-d209-4102-93b5-57a5fe8bfb94 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:35.587847 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 598/1174] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:22:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:22:35.593101 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2326b53-880b-4a1a-8992-0171f7b2f53e None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:35.593508 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2326b53-880b-4a1a-8992-0171f7b2f53e None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:35.593678 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2326b53-880b-4a1a-8992-0171f7b2f53e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:35.612268 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-c2326b53-880b-4a1a-8992-0171f7b2f53e None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:22:35.613448 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-c2326b53-880b-4a1a-8992-0171f7b2f53e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:22:35.670653 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:22:35.678695 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 577/1175] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:22:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 07:22:35.953037 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cb58b1-d209-4102-93b5-57a5fe8bfb94 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:22:36.056717 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cb58b1-d209-4102-93b5-57a5fe8bfb94 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:22:36.072553 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:22:36.089892 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366b4d9a-8927-475f-b024-3c7932564a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:22:36.090521 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366b4d9a-8927-475f-b024-3c7932564a4b None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:36.090802 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366b4d9a-8927-475f-b024-3c7932564a4b None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:36.091079 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366b4d9a-8927-475f-b024-3c7932564a4b None admin] PATH_INFO: `/v3/domains/b44c9c7b1b624302ab672b506886a69b` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:36.092185 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366b4d9a-8927-475f-b024-3c7932564a4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=b44c9c7b1b624302ab672b506886a69b)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:22:36.093645 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366b4d9a-8927-475f-b024-3c7932564a4b None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:22:36.095410 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 599/1176] 10.222.0.42 () {56 vars in 1183 bytes} [Thu Sep 15 07:22:35 2022] GET /identity/v3/domains/b44c9c7b1b624302ab672b506886a69b => generated 267 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:22:36.101650 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2326b53-880b-4a1a-8992-0171f7b2f53e None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:22:36.197100 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2326b53-880b-4a1a-8992-0171f7b2f53e None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:22:36.229707 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7c2506-4543-44ee-9de8-96e5d002c581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:22:36.230306 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7c2506-4543-44ee-9de8-96e5d002c581 None admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:36.230587 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7c2506-4543-44ee-9de8-96e5d002c581 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:36.230862 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7c2506-4543-44ee-9de8-96e5d002c581 None admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:36.231504 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7c2506-4543-44ee-9de8-96e5d002c581 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:22:36.232906 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7c2506-4543-44ee-9de8-96e5d002c581 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:22:36.337723 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 578/1177] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Sep 15 07:22:36 2022] POST /identity/v3/users => generated 311 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:22:38.532253 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff9bf7-abb8-4bd5-af49-d0e6bfd7409c None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:38.532782 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff9bf7-abb8-4bd5-af49-d0e6bfd7409c None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:38.532782 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff9bf7-abb8-4bd5-af49-d0e6bfd7409c None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:38.534190 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 600/1178] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:22:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:22:38.539614 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd56bac-2e42-4caf-909e-986996ea5dfe None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:38.540034 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd56bac-2e42-4caf-909e-986996ea5dfe None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:38.540140 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd56bac-2e42-4caf-909e-986996ea5dfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:38.558498 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-9fd56bac-2e42-4caf-909e-986996ea5dfe None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:22:38.559661 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-9fd56bac-2e42-4caf-909e-986996ea5dfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:22:38.604267 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:22:38.624585 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 579/1179] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:22:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 07:22:38.893002 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fff9bf7-abb8-4bd5-af49-d0e6bfd7409c None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:22:38.994810 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fff9bf7-abb8-4bd5-af49-d0e6bfd7409c None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:22:39.028049 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2619b8c0-c413-4f73-97c0-c1a75c987033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:22:39.028670 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2619b8c0-c413-4f73-97c0-c1a75c987033 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:39.028951 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2619b8c0-c413-4f73-97c0-c1a75c987033 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:39.029226 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2619b8c0-c413-4f73-97c0-c1a75c987033 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:39.039681 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2619b8c0-c413-4f73-97c0-c1a75c987033 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:22:39.044439 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-2619b8c0-c413-4f73-97c0-c1a75c987033 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 07:22:39.044439 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:22:39.044439 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:22:39.044439 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:22:39.044439 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:22:39.044439 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:22:39.044439 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:22:39.044439 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:22:39.044439 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:22:39.044439 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:22:39.044439 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:22:39.044439 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:22:39.044439 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:22:39.044439 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:22:39.044439 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:22:39.044439 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:22:39.045515 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:22:39.045515 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:22:39.045515 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:22:39.045515 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:22:39.045515 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:22:39.045515 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:22:39.045515 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:22:39.045515 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:22:39.045515 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:22:39.045515 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:22:39.045515 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:22:39.045515 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:22:39.045515 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:22:39.045515 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:22:39.045515 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:22:39.045515 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:22:39.046735 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:22:39.046735 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:22:39.046735 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:22:39.046735 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:22:39.046735 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:22:39.046735 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:22:39.046735 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:22:39.046735 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:22:39.046735 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:22:39.046735 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:22:39.046735 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 07:22:39.046735 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:22:39.046735 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 601/1180] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 15 07:22:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:22:39.051612 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd56bac-2e42-4caf-909e-986996ea5dfe None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:22:39.146841 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd56bac-2e42-4caf-909e-986996ea5dfe None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:22:39.178887 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0fc973-0ffe-40b2-a5ac-3c7682b73d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:22:39.179482 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0fc973-0ffe-40b2-a5ac-3c7682b73d10 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:39.179759 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0fc973-0ffe-40b2-a5ac-3c7682b73d10 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:39.180030 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0fc973-0ffe-40b2-a5ac-3c7682b73d10 None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:39.188044 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0fc973-0ffe-40b2-a5ac-3c7682b73d10 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:22:39.188044 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0fc973-0ffe-40b2-a5ac-3c7682b73d10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:22:39.188044 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0fc973-0ffe-40b2-a5ac-3c7682b73d10 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:22:39.190109 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 580/1181] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 15 07:22:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:22:39.196702 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2619b8c0-c413-4f73-97c0-c1a75c987033 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:22:39.283998 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:22:39.338134 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2619b8c0-c413-4f73-97c0-c1a75c987033 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:22:39.370607 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde08015-0d01-4270-9cce-4b4d776bf6fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:22:39.371228 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde08015-0d01-4270-9cce-4b4d776bf6fa None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:39.371505 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde08015-0d01-4270-9cce-4b4d776bf6fa None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:39.371777 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde08015-0d01-4270-9cce-4b4d776bf6fa None admin] PATH_INFO: `/v3/users/7301fc2ec4444392a97747f5f3339807` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:39.385001 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde08015-0d01-4270-9cce-4b4d776bf6fa None admin] RBAC: Authorizing `identity:get_user(user_id=7301fc2ec4444392a97747f5f3339807)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:22:39.386466 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde08015-0d01-4270-9cce-4b4d776bf6fa None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:22:39.388059 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 602/1182] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 15 07:22:39 2022] GET /identity/v3/users/7301fc2ec4444392a97747f5f3339807 => generated 311 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:22:39.393869 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0fc973-0ffe-40b2-a5ac-3c7682b73d10 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:22:39.436114 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:22:39.492375 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0fc973-0ffe-40b2-a5ac-3c7682b73d10 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:22:39.525039 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f11a527-6b9a-4898-8141-cc2cf5ab98f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:22:39.525665 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f11a527-6b9a-4898-8141-cc2cf5ab98f8 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:39.525944 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f11a527-6b9a-4898-8141-cc2cf5ab98f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:39.526211 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f11a527-6b9a-4898-8141-cc2cf5ab98f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:39.590891 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f11a527-6b9a-4898-8141-cc2cf5ab98f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:22:39.592307 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f11a527-6b9a-4898-8141-cc2cf5ab98f8 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:22:39.631769 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:22:39.686212 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 581/1183] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 15 07:22:39 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 07:22:39.693350 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde08015-0d01-4270-9cce-4b4d776bf6fa None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:22:39.796872 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde08015-0d01-4270-9cce-4b4d776bf6fa None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:22:39.798594 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:22:39.831669 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083d9688-3d02-4634-b491-f7483e6a1a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:22:39.832295 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083d9688-3d02-4634-b491-f7483e6a1a6a None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:39.833223 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083d9688-3d02-4634-b491-f7483e6a1a6a None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:39.833503 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083d9688-3d02-4634-b491-f7483e6a1a6a None admin] PATH_INFO: `/v3/projects/c85aead8f0b245f28d21c60ecda7ec0a` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:39.834718 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083d9688-3d02-4634-b491-f7483e6a1a6a None admin] RBAC: Authorizing `identity:get_project(project_id=c85aead8f0b245f28d21c60ecda7ec0a)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:22:39.836182 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083d9688-3d02-4634-b491-f7483e6a1a6a None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:22:39.837869 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 603/1184] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 15 07:22:39 2022] GET /identity/v3/projects/c85aead8f0b245f28d21c60ecda7ec0a => generated 361 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:22:39.844102 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f11a527-6b9a-4898-8141-cc2cf5ab98f8 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:22:39.938362 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f11a527-6b9a-4898-8141-cc2cf5ab98f8 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:22:39.972057 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafe44a0-42c6-41b0-8fb0-93bade978eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:22:39.972652 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafe44a0-42c6-41b0-8fb0-93bade978eab None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:39.972927 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafe44a0-42c6-41b0-8fb0-93bade978eab None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:39.973787 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafe44a0-42c6-41b0-8fb0-93bade978eab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:39.974670 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafe44a0-42c6-41b0-8fb0-93bade978eab None admin] RBAC: Adding query filter params (role.id=6bbb643e0d8d4bc0a9b27c26edf59216, scope.project.id=c85aead8f0b245f28d21c60ecda7ec0a, user.id=7301fc2ec4444392a97747f5f3339807) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:22:39.974974 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafe44a0-42c6-41b0-8fb0-93bade978eab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:22:39.976384 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafe44a0-42c6-41b0-8fb0-93bade978eab None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:22:39.983050 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 582/1185] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 15 07:22:39 2022] GET /identity/v3/role_assignments?user.id=7301fc2ec4444392a97747f5f3339807&scope.project.id=c85aead8f0b245f28d21c60ecda7ec0a&role.id=6bbb643e0d8d4bc0a9b27c26edf59216 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:22:42.109261 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06932055-8075-4547-a2c0-640e83792ebc None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:42.109831 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06932055-8075-4547-a2c0-640e83792ebc None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:42.109920 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06932055-8075-4547-a2c0-640e83792ebc None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:42.111184 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 604/1186] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:22:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:22:42.116860 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85e79e3-4061-4d5c-8a0b-9ad67dc3d1f4 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:42.117293 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85e79e3-4061-4d5c-8a0b-9ad67dc3d1f4 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:42.117412 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85e79e3-4061-4d5c-8a0b-9ad67dc3d1f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:42.130769 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:22:42.136179 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-c85e79e3-4061-4d5c-8a0b-9ad67dc3d1f4 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:22:42.137367 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-c85e79e3-4061-4d5c-8a0b-9ad67dc3d1f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:22:42.203243 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 583/1187] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:22:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 07:22:42.481146 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06932055-8075-4547-a2c0-640e83792ebc None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:22:42.580225 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06932055-8075-4547-a2c0-640e83792ebc None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:22:42.613378 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9796581-e38a-4ed6-af87-6a75a8a88a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:22:42.614046 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9796581-e38a-4ed6-af87-6a75a8a88a2a None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:42.614323 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9796581-e38a-4ed6-af87-6a75a8a88a2a None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:42.614597 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9796581-e38a-4ed6-af87-6a75a8a88a2a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:42.624517 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9796581-e38a-4ed6-af87-6a75a8a88a2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:22:42.629274 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-c9796581-e38a-4ed6-af87-6a75a8a88a2a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 07:22:42.629274 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:22:42.629274 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:22:42.629274 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:22:42.629274 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:22:42.629274 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:22:42.629274 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:22:42.629274 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:22:42.629274 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:22:42.629274 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:22:42.629274 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:22:42.629274 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:22:42.629274 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:22:42.629274 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:22:42.629274 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:22:42.629274 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:22:42.630648 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:22:42.630648 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:22:42.630648 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:22:42.630648 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:22:42.630648 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:22:42.630648 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:22:42.630648 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:22:42.630648 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:22:42.630648 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:22:42.630648 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:22:42.630648 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:22:42.630648 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:22:42.630648 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:22:42.630648 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:22:42.630648 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:22:42.630648 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:22:42.631644 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:22:42.631644 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:22:42.631644 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:22:42.631644 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:22:42.631644 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:22:42.631644 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:22:42.631644 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:22:42.631644 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:22:42.631644 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:22:42.631644 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:22:42.631644 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 07:22:42.631644 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:22:42.631644 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 605/1188] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 15 07:22:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:22:42.636515 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85e79e3-4061-4d5c-8a0b-9ad67dc3d1f4 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:22:42.732271 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85e79e3-4061-4d5c-8a0b-9ad67dc3d1f4 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:22:42.766313 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b68a2e-eb55-43cc-9473-894269da36b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:22:42.766918 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b68a2e-eb55-43cc-9473-894269da36b5 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:42.767198 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b68a2e-eb55-43cc-9473-894269da36b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:42.767469 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b68a2e-eb55-43cc-9473-894269da36b5 None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:42.768130 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b68a2e-eb55-43cc-9473-894269da36b5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:22:42.768432 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b68a2e-eb55-43cc-9473-894269da36b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:22:42.769818 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b68a2e-eb55-43cc-9473-894269da36b5 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:22:42.772956 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:22:42.779401 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 584/1189] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 15 07:22:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:22:42.785244 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9796581-e38a-4ed6-af87-6a75a8a88a2a None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:22:42.821024 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:22:42.892454 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9796581-e38a-4ed6-af87-6a75a8a88a2a None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:22:42.926010 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b6a774-535f-45fc-b64c-960ab04df3c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:22:42.926828 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b6a774-535f-45fc-b64c-960ab04df3c3 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:42.927212 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b6a774-535f-45fc-b64c-960ab04df3c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:42.927493 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b6a774-535f-45fc-b64c-960ab04df3c3 None admin] PATH_INFO: `/v3/users/7301fc2ec4444392a97747f5f3339807` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:42.928606 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b6a774-535f-45fc-b64c-960ab04df3c3 None admin] RBAC: Authorizing `identity:get_user(user_id=7301fc2ec4444392a97747f5f3339807)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:22:42.930069 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b6a774-535f-45fc-b64c-960ab04df3c3 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:22:42.931629 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 606/1190] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 15 07:22:42 2022] GET /identity/v3/users/7301fc2ec4444392a97747f5f3339807 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:22:42.937430 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b68a2e-eb55-43cc-9473-894269da36b5 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:22:43.036706 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b68a2e-eb55-43cc-9473-894269da36b5 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:22:43.072792 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28418b03-5790-4d2e-b88e-a2c04d9e21c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:22:43.073430 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28418b03-5790-4d2e-b88e-a2c04d9e21c6 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:43.073749 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28418b03-5790-4d2e-b88e-a2c04d9e21c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:43.074025 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28418b03-5790-4d2e-b88e-a2c04d9e21c6 None admin] PATH_INFO: `/v3/projects/c85aead8f0b245f28d21c60ecda7ec0a` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:43.075143 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28418b03-5790-4d2e-b88e-a2c04d9e21c6 None admin] RBAC: Authorizing `identity:get_project(project_id=c85aead8f0b245f28d21c60ecda7ec0a)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:22:43.076568 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28418b03-5790-4d2e-b88e-a2c04d9e21c6 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:22:43.080670 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 585/1191] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 15 07:22:42 2022] GET /identity/v3/projects/c85aead8f0b245f28d21c60ecda7ec0a => generated 361 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:22:43.087089 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b6a774-535f-45fc-b64c-960ab04df3c3 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:22:43.180602 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:22:43.184411 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b6a774-535f-45fc-b64c-960ab04df3c3 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:22:43.218509 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85845fb1-7025-44a5-961f-f401f9989ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:22:43.219153 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85845fb1-7025-44a5-961f-f401f9989ce7 None admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:43.219432 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85845fb1-7025-44a5-961f-f401f9989ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:43.219705 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85845fb1-7025-44a5-961f-f401f9989ce7 None admin] PATH_INFO: `/v3/projects/c85aead8f0b245f28d21c60ecda7ec0a/users/7301fc2ec4444392a97747f5f3339807/roles/6bbb643e0d8d4bc0a9b27c26edf59216` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:43.221549 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85845fb1-7025-44a5-961f-f401f9989ce7 None admin] RBAC: Authorizing `identity:create_grant(project_id=c85aead8f0b245f28d21c60ecda7ec0a, user_id=7301fc2ec4444392a97747f5f3339807, role_id=6bbb643e0d8d4bc0a9b27c26edf59216)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:22:43.222996 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85845fb1-7025-44a5-961f-f401f9989ce7 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:22:43.292335 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 607/1192] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Sep 15 07:22:43 2022] PUT /identity/v3/projects/c85aead8f0b245f28d21c60ecda7ec0a/users/7301fc2ec4444392a97747f5f3339807/roles/6bbb643e0d8d4bc0a9b27c26edf59216 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:22:45.402430 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914db502-7e00-4311-a29a-62559a6cebc8 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:45.402959 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914db502-7e00-4311-a29a-62559a6cebc8 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:45.402959 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914db502-7e00-4311-a29a-62559a6cebc8 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:45.404307 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 586/1193] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:22:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:22:45.409601 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3506a6d-7880-4d07-b0f1-77dec0180235 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:45.410016 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3506a6d-7880-4d07-b0f1-77dec0180235 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:45.410176 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3506a6d-7880-4d07-b0f1-77dec0180235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:45.428655 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-b3506a6d-7880-4d07-b0f1-77dec0180235 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:22:45.429864 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-b3506a6d-7880-4d07-b0f1-77dec0180235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:22:45.496132 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 608/1194] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:22:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 07:22:45.763508 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914db502-7e00-4311-a29a-62559a6cebc8 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:22:45.829241 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:22:45.864092 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914db502-7e00-4311-a29a-62559a6cebc8 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:22:45.897069 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563483c6-c431-44ff-909d-7f90cd2426d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:22:45.897707 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563483c6-c431-44ff-909d-7f90cd2426d5 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:45.897986 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563483c6-c431-44ff-909d-7f90cd2426d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:45.898257 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563483c6-c431-44ff-909d-7f90cd2426d5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:45.908467 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563483c6-c431-44ff-909d-7f90cd2426d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:22:45.913266 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-563483c6-c431-44ff-909d-7f90cd2426d5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 07:22:45.913266 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:22:45.913266 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:22:45.913266 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:22:45.913266 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:22:45.913266 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:22:45.913266 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:22:45.913266 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:22:45.913266 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:22:45.913266 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:22:45.913266 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:22:45.913266 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:22:45.913266 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:22:45.913266 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:22:45.913266 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:22:45.913266 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:22:45.914395 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:22:45.914395 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:22:45.914395 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:22:45.914395 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:22:45.914395 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:22:45.914395 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:22:45.914395 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:22:45.914395 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:22:45.914395 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:22:45.914395 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:22:45.914395 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:22:45.914395 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:22:45.914395 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:22:45.914395 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:22:45.914395 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:22:45.914395 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:22:45.915267 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:22:45.915267 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:22:45.915267 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:22:45.915267 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:22:45.915267 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:22:45.915267 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:22:45.915267 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:22:45.915267 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:22:45.915267 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:22:45.915267 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:22:45.915267 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 07:22:45.915267 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:22:45.915267 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 587/1195] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 15 07:22:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:22:45.920322 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3506a6d-7880-4d07-b0f1-77dec0180235 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:22:46.005941 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:22:46.017833 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3506a6d-7880-4d07-b0f1-77dec0180235 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:22:46.057190 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0758686-92b6-4d26-9216-aaeb6f3ac666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:22:46.057829 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0758686-92b6-4d26-9216-aaeb6f3ac666 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:46.058111 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0758686-92b6-4d26-9216-aaeb6f3ac666 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:46.058385 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0758686-92b6-4d26-9216-aaeb6f3ac666 None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:46.059044 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0758686-92b6-4d26-9216-aaeb6f3ac666 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:22:46.059349 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0758686-92b6-4d26-9216-aaeb6f3ac666 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:22:46.060714 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0758686-92b6-4d26-9216-aaeb6f3ac666 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:22:46.070555 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 609/1196] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 15 07:22:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:22:46.076506 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563483c6-c431-44ff-909d-7f90cd2426d5 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:22:46.171570 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563483c6-c431-44ff-909d-7f90cd2426d5 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:22:46.196939 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:22:46.204912 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2d67d7-210a-4391-9980-978920b0523a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:22:46.205526 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2d67d7-210a-4391-9980-978920b0523a None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:46.205835 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2d67d7-210a-4391-9980-978920b0523a None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:46.206102 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2d67d7-210a-4391-9980-978920b0523a None admin] PATH_INFO: `/v3/users/7301fc2ec4444392a97747f5f3339807` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:46.207189 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2d67d7-210a-4391-9980-978920b0523a None admin] RBAC: Authorizing `identity:get_user(user_id=7301fc2ec4444392a97747f5f3339807)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:22:46.208618 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2d67d7-210a-4391-9980-978920b0523a None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:22:46.210242 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 588/1197] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 15 07:22:46 2022] GET /identity/v3/users/7301fc2ec4444392a97747f5f3339807 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:22:46.216025 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0758686-92b6-4d26-9216-aaeb6f3ac666 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:22:46.315808 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0758686-92b6-4d26-9216-aaeb6f3ac666 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:22:46.357328 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c76427-1a38-43ff-b2e7-ba9b56518976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:22:46.357953 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c76427-1a38-43ff-b2e7-ba9b56518976 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:46.358232 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c76427-1a38-43ff-b2e7-ba9b56518976 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:46.358502 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c76427-1a38-43ff-b2e7-ba9b56518976 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:46.376745 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:22:46.436193 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c76427-1a38-43ff-b2e7-ba9b56518976 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:22:46.437786 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c76427-1a38-43ff-b2e7-ba9b56518976 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:22:46.536839 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 610/1198] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 15 07:22:46 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 07:22:46.542718 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2d67d7-210a-4391-9980-978920b0523a None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:22:46.641158 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2d67d7-210a-4391-9980-978920b0523a None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:22:46.673426 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9935225-bd87-4acf-bada-1ec743a03c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:22:46.674062 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9935225-bd87-4acf-bada-1ec743a03c4f None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:46.674342 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9935225-bd87-4acf-bada-1ec743a03c4f None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:46.674611 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9935225-bd87-4acf-bada-1ec743a03c4f None admin] PATH_INFO: `/v3/projects/c85aead8f0b245f28d21c60ecda7ec0a` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:46.675705 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9935225-bd87-4acf-bada-1ec743a03c4f None admin] RBAC: Authorizing `identity:get_project(project_id=c85aead8f0b245f28d21c60ecda7ec0a)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:22:46.677753 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9935225-bd87-4acf-bada-1ec743a03c4f None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:22:46.679360 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 589/1199] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 15 07:22:46 2022] GET /identity/v3/projects/c85aead8f0b245f28d21c60ecda7ec0a => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:22:46.685568 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c76427-1a38-43ff-b2e7-ba9b56518976 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:22:46.738151 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:22:46.787496 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c76427-1a38-43ff-b2e7-ba9b56518976 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:22:46.821012 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a2235a-6913-4589-a8a4-e623a3480718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:22:46.821637 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a2235a-6913-4589-a8a4-e623a3480718 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:22:46.821920 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a2235a-6913-4589-a8a4-e623a3480718 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:22:46.822191 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a2235a-6913-4589-a8a4-e623a3480718 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:22:46.822907 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a2235a-6913-4589-a8a4-e623a3480718 None admin] RBAC: Adding query filter params (role.id=6bbb643e0d8d4bc0a9b27c26edf59216, scope.project.id=c85aead8f0b245f28d21c60ecda7ec0a, user.id=7301fc2ec4444392a97747f5f3339807) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:22:46.823213 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a2235a-6913-4589-a8a4-e623a3480718 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:22:46.824639 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a2235a-6913-4589-a8a4-e623a3480718 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:22:46.830335 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 611/1200] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 15 07:22:46 2022] GET /identity/v3/role_assignments?user.id=7301fc2ec4444392a97747f5f3339807&scope.project.id=c85aead8f0b245f28d21c60ecda7ec0a&role.id=6bbb643e0d8d4bc0a9b27c26edf59216 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:22:56.669321 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=69 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:22:56.827830 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=43 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:23:59.341629 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555626c3-4022-4c48-a3ca-211af76b0178 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:23:59.342206 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555626c3-4022-4c48-a3ca-211af76b0178 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:23:59.342206 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555626c3-4022-4c48-a3ca-211af76b0178 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:23:59.343506 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 590/1201] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:23:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:23:59.348733 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f798a0ff-5080-4fac-ad8d-273c66b64c8a None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:23:59.349172 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f798a0ff-5080-4fac-ad8d-273c66b64c8a None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:23:59.349301 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f798a0ff-5080-4fac-ad8d-273c66b64c8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:23:59.367506 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-f798a0ff-5080-4fac-ad8d-273c66b64c8a None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:23:59.368697 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-f798a0ff-5080-4fac-ad8d-273c66b64c8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:23:59.435565 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 612/1202] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:23:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 07:23:59.720219 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555626c3-4022-4c48-a3ca-211af76b0178 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:23:59.833764 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555626c3-4022-4c48-a3ca-211af76b0178 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:23:59.867338 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccaa0cd-9999-4e4f-b559-1885c0c920bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:23:59.867959 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccaa0cd-9999-4e4f-b559-1885c0c920bf None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:23:59.868238 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccaa0cd-9999-4e4f-b559-1885c0c920bf None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:23:59.868513 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccaa0cd-9999-4e4f-b559-1885c0c920bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:23:59.879093 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccaa0cd-9999-4e4f-b559-1885c0c920bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:23:59.880554 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccaa0cd-9999-4e4f-b559-1885c0c920bf None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:23:59.895254 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-3ccaa0cd-9999-4e4f-b559-1885c0c920bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:23:59.895254 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:23:59.895254 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:23:59.895254 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:23:59.895254 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:23:59.895254 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:23:59.895254 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:23:59.895254 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:23:59.895254 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:23:59.895254 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:23:59.895254 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:23:59.895254 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:23:59.895254 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 07:23:59.895254 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 07:23:59.895254 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 07:23:59.895254 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 07:23:59.896483 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:23:59.896483 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:23:59.896483 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:23:59.896483 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:23:59.896483 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:23:59.896483 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:23:59.896483 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:23:59.896483 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:23:59.896483 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:23:59.896483 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:23:59.896483 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:23:59.896483 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:23:59.896483 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:23:59.896483 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:23:59.896483 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:23:59.896483 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:23:59.896483 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 07:23:59.897665 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 07:23:59.897665 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:23:59.897665 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:23:59.897665 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 591/1203] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:23:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:23:59.902704 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f798a0ff-5080-4fac-ad8d-273c66b64c8a None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:24:00.002073 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f798a0ff-5080-4fac-ad8d-273c66b64c8a None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:24:00.029520 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:24:00.035276 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b7f72f-80f8-4611-8bd5-d933391334fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:24:00.035876 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b7f72f-80f8-4611-8bd5-d933391334fa None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:00.036154 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b7f72f-80f8-4611-8bd5-d933391334fa None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:00.036425 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b7f72f-80f8-4611-8bd5-d933391334fa None admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:00.037054 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b7f72f-80f8-4611-8bd5-d933391334fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:24:00.037355 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b7f72f-80f8-4611-8bd5-d933391334fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:24:00.038784 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b7f72f-80f8-4611-8bd5-d933391334fa None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:24:00.050240 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 613/1204] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 15 07:23:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:24:00.055993 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccaa0cd-9999-4e4f-b559-1885c0c920bf None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:24:00.110142 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:24:00.156916 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccaa0cd-9999-4e4f-b559-1885c0c920bf None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:24:00.190798 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692de749-b981-4aaf-9a10-bef2b913c9b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:24:00.191526 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692de749-b981-4aaf-9a10-bef2b913c9b9 None admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:00.191804 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692de749-b981-4aaf-9a10-bef2b913c9b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:00.192070 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692de749-b981-4aaf-9a10-bef2b913c9b9 None admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:00.192708 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692de749-b981-4aaf-9a10-bef2b913c9b9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:24:00.194145 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692de749-b981-4aaf-9a10-bef2b913c9b9 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:24:00.265380 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 592/1205] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Sep 15 07:24:00 2022] POST /identity/v3/users => generated 251 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:24:02.486181 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa8a198-8c3a-4b2e-838f-b847b09f002d None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:02.486723 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa8a198-8c3a-4b2e-838f-b847b09f002d None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:02.486723 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa8a198-8c3a-4b2e-838f-b847b09f002d None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:02.488076 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 614/1206] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:24:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:24:02.493486 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaddfcf-3e0e-42c8-a51d-7df114b6c00c None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:02.493887 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaddfcf-3e0e-42c8-a51d-7df114b6c00c None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:02.494068 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaddfcf-3e0e-42c8-a51d-7df114b6c00c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:02.512754 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-fcaddfcf-3e0e-42c8-a51d-7df114b6c00c None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:24:02.518103 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-fcaddfcf-3e0e-42c8-a51d-7df114b6c00c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:24:02.581222 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 593/1207] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:24:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 07:24:02.858308 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa8a198-8c3a-4b2e-838f-b847b09f002d None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:24:02.960468 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa8a198-8c3a-4b2e-838f-b847b09f002d None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:24:02.994112 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41835946-9b44-4d0c-a354-37c281bd5a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:24:02.994817 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41835946-9b44-4d0c-a354-37c281bd5a2b None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:02.995645 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41835946-9b44-4d0c-a354-37c281bd5a2b None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:02.995898 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41835946-9b44-4d0c-a354-37c281bd5a2b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:03.007299 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41835946-9b44-4d0c-a354-37c281bd5a2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:24:03.015359 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-41835946-9b44-4d0c-a354-37c281bd5a2b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 07:24:03.015359 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:24:03.015359 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:24:03.015359 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:24:03.015359 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:24:03.015359 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:24:03.015359 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:24:03.015359 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:24:03.015359 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:24:03.015359 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:24:03.015359 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:24:03.015359 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:24:03.015359 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:24:03.015359 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:24:03.015359 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:24:03.015359 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:24:03.016468 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:24:03.016468 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:24:03.016468 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:24:03.016468 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:24:03.016468 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:24:03.016468 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:24:03.016468 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:24:03.016468 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:24:03.016468 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:24:03.016468 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:24:03.016468 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:24:03.016468 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:24:03.016468 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:24:03.016468 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:24:03.016468 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:24:03.016468 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:24:03.017511 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:24:03.017511 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:24:03.017511 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:24:03.017511 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:24:03.017511 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:24:03.017511 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:24:03.017511 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:24:03.017511 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:24:03.017511 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:24:03.017511 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:24:03.017511 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 07:24:03.017511 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:24:03.017511 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 615/1208] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Sep 15 07:24:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:24:03.019292 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaddfcf-3e0e-42c8-a51d-7df114b6c00c None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:24:03.061235 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:24:03.130574 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaddfcf-3e0e-42c8-a51d-7df114b6c00c None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:24:03.163352 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b45d2c-291d-44f1-9669-06e0f16a7dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:24:03.163950 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b45d2c-291d-44f1-9669-06e0f16a7dd6 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:03.164230 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b45d2c-291d-44f1-9669-06e0f16a7dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:03.164497 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b45d2c-291d-44f1-9669-06e0f16a7dd6 None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:03.165147 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b45d2c-291d-44f1-9669-06e0f16a7dd6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:24:03.165444 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b45d2c-291d-44f1-9669-06e0f16a7dd6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:24:03.167329 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b45d2c-291d-44f1-9669-06e0f16a7dd6 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:24:03.174906 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 594/1209] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Sep 15 07:24:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:24:03.180765 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41835946-9b44-4d0c-a354-37c281bd5a2b None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:24:03.232243 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:24:03.281886 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41835946-9b44-4d0c-a354-37c281bd5a2b None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:24:03.315844 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef23d65-6c75-4573-9a8f-5801f6f91d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:24:03.316549 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef23d65-6c75-4573-9a8f-5801f6f91d87 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:03.316827 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef23d65-6c75-4573-9a8f-5801f6f91d87 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:03.317093 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef23d65-6c75-4573-9a8f-5801f6f91d87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:03.327935 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef23d65-6c75-4573-9a8f-5801f6f91d87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:24:03.329396 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef23d65-6c75-4573-9a8f-5801f6f91d87 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:24:03.344151 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-5ef23d65-6c75-4573-9a8f-5801f6f91d87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:24:03.344151 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:24:03.344151 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:24:03.344151 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:24:03.344151 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:24:03.344151 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:24:03.344151 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:24:03.344151 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:24:03.344151 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:24:03.344151 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:24:03.344151 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:24:03.344151 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:24:03.344151 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 07:24:03.344151 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 07:24:03.344151 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 07:24:03.344151 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 07:24:03.345290 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:24:03.345290 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:24:03.345290 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:24:03.345290 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:24:03.345290 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:24:03.345290 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:24:03.345290 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:24:03.345290 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:24:03.345290 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:24:03.345290 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:24:03.345290 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:24:03.345290 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:24:03.345290 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:24:03.345290 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:24:03.345290 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:24:03.345290 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:24:03.345290 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 07:24:03.346405 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 07:24:03.346405 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:24:03.346405 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:24:03.346405 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 616/1210] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:24:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:24:03.351015 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b45d2c-291d-44f1-9669-06e0f16a7dd6 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:24:03.449643 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:24:03.452553 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b45d2c-291d-44f1-9669-06e0f16a7dd6 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:24:03.486338 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46041c5c-5df9-4b00-a639-c78cd98d07ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:24:03.486935 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46041c5c-5df9-4b00-a639-c78cd98d07ed None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:03.487216 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46041c5c-5df9-4b00-a639-c78cd98d07ed None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:03.487486 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46041c5c-5df9-4b00-a639-c78cd98d07ed None admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:03.488117 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46041c5c-5df9-4b00-a639-c78cd98d07ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:24:03.488416 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46041c5c-5df9-4b00-a639-c78cd98d07ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:24:03.489786 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46041c5c-5df9-4b00-a639-c78cd98d07ed None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:24:03.501005 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 595/1211] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 15 07:24:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:24:03.506753 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef23d65-6c75-4573-9a8f-5801f6f91d87 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:24:03.605792 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef23d65-6c75-4573-9a8f-5801f6f91d87 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:24:03.607304 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:24:03.638902 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a476690-3e21-4b8d-a9a2-78bb76dfdc1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:24:03.639523 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a476690-3e21-4b8d-a9a2-78bb76dfdc1f None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:03.639799 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a476690-3e21-4b8d-a9a2-78bb76dfdc1f None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:03.640070 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a476690-3e21-4b8d-a9a2-78bb76dfdc1f None admin] PATH_INFO: `/v3/users/placement` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:03.647076 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a476690-3e21-4b8d-a9a2-78bb76dfdc1f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:24:03.648496 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a476690-3e21-4b8d-a9a2-78bb76dfdc1f None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:24:03.659399 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-3a476690-3e21-4b8d-a9a2-78bb76dfdc1f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 15 07:24:03.659399 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:24:03.659399 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:24:03.659399 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:24:03.659399 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:24:03.659399 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:24:03.659399 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:24:03.659399 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:24:03.659399 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:24:03.659399 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:24:03.659399 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:24:03.659399 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:24:03.659399 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 07:24:03.659399 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 07:24:03.659399 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 07:24:03.659399 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 07:24:03.660554 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:24:03.660554 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:24:03.660554 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 07:24:03.660554 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 07:24:03.660554 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 07:24:03.660554 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 07:24:03.660554 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:24:03.660554 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:24:03.660554 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:24:03.660554 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:24:03.660554 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:24:03.660554 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:24:03.660554 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:24:03.660554 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:24:03.660554 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:24:03.660554 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:24:03.660554 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:24:03.660554 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:24:03.661516 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:24:03.661516 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:24:03.661516 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 07:24:03.661516 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 07:24:03.661516 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 07:24:03.661516 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 07:24:03.661516 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 07:24:03.661516 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 07:24:03.661516 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 15 07:24:03.661516 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:24:03.661516 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 617/1212] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:24:03 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:24:03.666482 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46041c5c-5df9-4b00-a639-c78cd98d07ed None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:24:03.768853 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46041c5c-5df9-4b00-a639-c78cd98d07ed None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:24:03.803590 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61a3279-f81f-45d5-ad79-5e1b168ae5b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:24:03.804223 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61a3279-f81f-45d5-ad79-5e1b168ae5b8 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:03.804504 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61a3279-f81f-45d5-ad79-5e1b168ae5b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:03.804776 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61a3279-f81f-45d5-ad79-5e1b168ae5b8 None admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:03.805494 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61a3279-f81f-45d5-ad79-5e1b168ae5b8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:24:03.805828 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61a3279-f81f-45d5-ad79-5e1b168ae5b8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:24:03.807230 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61a3279-f81f-45d5-ad79-5e1b168ae5b8 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:24:03.827517 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 596/1213] 10.222.0.42 () {56 vars in 1143 bytes} [Thu Sep 15 07:24:03 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:24:03.833295 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a476690-3e21-4b8d-a9a2-78bb76dfdc1f None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:24:03.932929 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a476690-3e21-4b8d-a9a2-78bb76dfdc1f None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:24:03.967800 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1233843a-715b-49dc-84a1-a2b17de2a04c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:24:03.968413 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1233843a-715b-49dc-84a1-a2b17de2a04c None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:03.968696 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1233843a-715b-49dc-84a1-a2b17de2a04c None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:03.968966 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1233843a-715b-49dc-84a1-a2b17de2a04c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:03.999236 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:24:04.035407 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1233843a-715b-49dc-84a1-a2b17de2a04c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:24:04.036785 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1233843a-715b-49dc-84a1-a2b17de2a04c None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:24:04.137943 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 618/1214] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 15 07:24:03 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 07:24:04.143745 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61a3279-f81f-45d5-ad79-5e1b168ae5b8 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:24:04.163007 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:24:04.243073 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61a3279-f81f-45d5-ad79-5e1b168ae5b8 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:24:04.278325 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e98858d-b6bf-4c97-b589-7d6b4fe6a288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:24:04.279015 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e98858d-b6bf-4c97-b589-7d6b4fe6a288 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:04.279292 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e98858d-b6bf-4c97-b589-7d6b4fe6a288 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:04.279561 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e98858d-b6bf-4c97-b589-7d6b4fe6a288 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:04.289550 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e98858d-b6bf-4c97-b589-7d6b4fe6a288 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:24:04.290818 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e98858d-b6bf-4c97-b589-7d6b4fe6a288 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:24:04.305475 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-2e98858d-b6bf-4c97-b589-7d6b4fe6a288 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:24:04.305475 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:24:04.305475 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:24:04.305475 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:24:04.305475 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:24:04.305475 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:24:04.305475 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:24:04.305475 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:24:04.305475 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:24:04.305475 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:24:04.305475 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:24:04.305475 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:24:04.305475 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 07:24:04.305475 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 07:24:04.305475 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 07:24:04.305475 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 07:24:04.306676 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:24:04.306676 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:24:04.306676 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:24:04.306676 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:24:04.306676 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:24:04.306676 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:24:04.306676 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:24:04.306676 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:24:04.306676 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:24:04.306676 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:24:04.306676 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:24:04.306676 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:24:04.306676 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:24:04.306676 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:24:04.306676 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:24:04.306676 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:24:04.306676 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 07:24:04.307709 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 07:24:04.307709 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:24:04.307709 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:24:04.307709 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 597/1215] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:24:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:24:04.312430 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1233843a-715b-49dc-84a1-a2b17de2a04c None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:24:04.374711 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:24:04.413424 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1233843a-715b-49dc-84a1-a2b17de2a04c None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:24:04.445809 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c839dfb-0b40-4ce8-8751-7d0bf4dc17f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:24:04.446404 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c839dfb-0b40-4ce8-8751-7d0bf4dc17f2 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:04.446680 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c839dfb-0b40-4ce8-8751-7d0bf4dc17f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:04.446950 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c839dfb-0b40-4ce8-8751-7d0bf4dc17f2 None admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:04.447582 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c839dfb-0b40-4ce8-8751-7d0bf4dc17f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:24:04.447881 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c839dfb-0b40-4ce8-8751-7d0bf4dc17f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:24:04.450095 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c839dfb-0b40-4ce8-8751-7d0bf4dc17f2 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:24:04.461532 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 619/1216] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 15 07:24:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:24:04.468118 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e98858d-b6bf-4c97-b589-7d6b4fe6a288 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:24:04.538920 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:24:04.570388 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e98858d-b6bf-4c97-b589-7d6b4fe6a288 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:24:04.605261 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2770e1b0-5763-47f9-aaa5-e96018b9428a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:24:04.605915 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2770e1b0-5763-47f9-aaa5-e96018b9428a None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:04.606194 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2770e1b0-5763-47f9-aaa5-e96018b9428a None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:04.606460 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2770e1b0-5763-47f9-aaa5-e96018b9428a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:04.612979 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2770e1b0-5763-47f9-aaa5-e96018b9428a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:24:04.614415 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2770e1b0-5763-47f9-aaa5-e96018b9428a None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:24:04.625457 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-2770e1b0-5763-47f9-aaa5-e96018b9428a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 07:24:04.625457 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:24:04.625457 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:24:04.625457 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:24:04.625457 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:24:04.625457 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:24:04.625457 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:24:04.625457 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:24:04.625457 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:24:04.625457 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:24:04.625457 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:24:04.625457 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:24:04.625457 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 07:24:04.625457 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 07:24:04.625457 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 07:24:04.626599 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 07:24:04.626599 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:24:04.626599 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:24:04.626599 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:24:04.626599 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:24:04.626599 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:24:04.626599 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:24:04.626599 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:24:04.626599 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:24:04.626599 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:24:04.626599 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:24:04.626599 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:24:04.626599 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:24:04.626599 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:24:04.626599 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:24:04.626599 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:24:04.626599 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:24:04.627841 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 07:24:04.627841 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 07:24:04.627841 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 07:24:04.627841 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 07:24:04.627841 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 07:24:04.627841 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 07:24:04.627841 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 07:24:04.627841 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:24:04.627841 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 598/1217] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 15 07:24:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:24:04.632611 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c839dfb-0b40-4ce8-8751-7d0bf4dc17f2 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:24:04.735017 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c839dfb-0b40-4ce8-8751-7d0bf4dc17f2 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:24:04.757389 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:24:04.778093 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d094c728-64e4-4e9a-99fc-fe56fc833682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:24:04.778715 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d094c728-64e4-4e9a-99fc-fe56fc833682 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:04.778997 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d094c728-64e4-4e9a-99fc-fe56fc833682 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:04.779293 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d094c728-64e4-4e9a-99fc-fe56fc833682 None admin] PATH_INFO: `/v3/projects` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:04.779965 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d094c728-64e4-4e9a-99fc-fe56fc833682 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:24:04.780273 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d094c728-64e4-4e9a-99fc-fe56fc833682 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:24:04.781729 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d094c728-64e4-4e9a-99fc-fe56fc833682 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:24:04.795207 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 620/1218] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 15 07:24:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:24:04.801155 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2770e1b0-5763-47f9-aaa5-e96018b9428a None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:24:04.905088 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2770e1b0-5763-47f9-aaa5-e96018b9428a None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:24:04.932451 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:24:04.937913 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae3d643-4d78-4f11-aa3d-9e25b68a4fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:24:04.938514 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae3d643-4d78-4f11-aa3d-9e25b68a4fec None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:04.938795 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae3d643-4d78-4f11-aa3d-9e25b68a4fec None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:04.939103 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae3d643-4d78-4f11-aa3d-9e25b68a4fec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:04.939825 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae3d643-4d78-4f11-aa3d-9e25b68a4fec None admin] RBAC: Adding query filter params (role.id=0e661b5a67664692b933e5da9fc2e488, scope.project.id=4e9987720930473da840579654311bc2, user.id=dd87846b9a134248b28e0521b0a880c9) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:24:04.940129 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae3d643-4d78-4f11-aa3d-9e25b68a4fec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:24:04.941534 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae3d643-4d78-4f11-aa3d-9e25b68a4fec None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:24:04.946977 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 599/1219] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 15 07:24:04 2022] GET /identity/v3/role_assignments?user.id=dd87846b9a134248b28e0521b0a880c9&scope.project.id=4e9987720930473da840579654311bc2&role.id=0e661b5a67664692b933e5da9fc2e488 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:24:07.075993 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753e4122-3053-417e-8420-016772f7782f None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:07.076557 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753e4122-3053-417e-8420-016772f7782f None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:07.076648 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753e4122-3053-417e-8420-016772f7782f None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:07.077965 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 621/1220] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:24:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 07:24:07.083583 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a612dbaa-df37-4ffe-aef0-b38ad55ee1b0 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:07.084066 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a612dbaa-df37-4ffe-aef0-b38ad55ee1b0 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:07.084177 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a612dbaa-df37-4ffe-aef0-b38ad55ee1b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:07.103107 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-a612dbaa-df37-4ffe-aef0-b38ad55ee1b0 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:24:07.105073 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-a612dbaa-df37-4ffe-aef0-b38ad55ee1b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:24:07.171915 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 600/1221] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:24:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 07:24:07.436991 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753e4122-3053-417e-8420-016772f7782f None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:24:07.537384 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753e4122-3053-417e-8420-016772f7782f None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:24:07.571645 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab78e6c-4656-471c-a8ae-adc2796a9083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:24:07.572264 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab78e6c-4656-471c-a8ae-adc2796a9083 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:07.572541 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab78e6c-4656-471c-a8ae-adc2796a9083 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:07.572810 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab78e6c-4656-471c-a8ae-adc2796a9083 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:07.583781 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab78e6c-4656-471c-a8ae-adc2796a9083 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:24:07.588606 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-9ab78e6c-4656-471c-a8ae-adc2796a9083 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 07:24:07.588606 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:24:07.588606 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:24:07.588606 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:24:07.588606 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:24:07.588606 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:24:07.588606 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:24:07.588606 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:24:07.588606 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:24:07.588606 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:24:07.588606 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:24:07.588606 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:24:07.588606 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:24:07.588606 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:24:07.588606 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:24:07.588606 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:24:07.593642 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:24:07.593642 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:24:07.593642 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:24:07.593642 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:24:07.593642 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:24:07.593642 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:24:07.593642 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:24:07.593642 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:24:07.593642 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:24:07.593642 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:24:07.593642 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:24:07.593642 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:24:07.593642 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:24:07.593642 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:24:07.593642 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:24:07.593642 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:24:07.594730 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:24:07.594730 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:24:07.594730 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:24:07.594730 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:24:07.594730 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:24:07.594730 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:24:07.594730 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:24:07.594730 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:24:07.594730 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:24:07.594730 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:24:07.594730 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 07:24:07.594730 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:24:07.594730 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 622/1222] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Sep 15 07:24:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:24:07.596207 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a612dbaa-df37-4ffe-aef0-b38ad55ee1b0 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:24:07.695535 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a612dbaa-df37-4ffe-aef0-b38ad55ee1b0 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:24:07.718189 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:24:07.728547 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337965c7-65db-4d03-b85d-d6d5ff65254c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:24:07.729150 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337965c7-65db-4d03-b85d-d6d5ff65254c None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:07.729995 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337965c7-65db-4d03-b85d-d6d5ff65254c None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:07.729995 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337965c7-65db-4d03-b85d-d6d5ff65254c None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:07.730912 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337965c7-65db-4d03-b85d-d6d5ff65254c None admin] RBAC: Adding query filter params (name=service) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:24:07.731215 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337965c7-65db-4d03-b85d-d6d5ff65254c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:24:07.732566 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337965c7-65db-4d03-b85d-d6d5ff65254c None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:24:07.740257 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 601/1223] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Sep 15 07:24:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:24:07.746727 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab78e6c-4656-471c-a8ae-adc2796a9083 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:24:07.790843 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:24:07.847262 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab78e6c-4656-471c-a8ae-adc2796a9083 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:24:07.882141 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c4bdfe-826f-470b-9553-6181499005e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:24:07.882774 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c4bdfe-826f-470b-9553-6181499005e4 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:07.883058 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c4bdfe-826f-470b-9553-6181499005e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:07.883363 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c4bdfe-826f-470b-9553-6181499005e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:07.893866 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c4bdfe-826f-470b-9553-6181499005e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:24:07.895299 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c4bdfe-826f-470b-9553-6181499005e4 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:24:07.910654 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-b1c4bdfe-826f-470b-9553-6181499005e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:24:07.910654 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:24:07.910654 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:24:07.910654 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:24:07.910654 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:24:07.910654 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:24:07.910654 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:24:07.910654 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:24:07.910654 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:24:07.910654 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:24:07.910654 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:24:07.910654 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:24:07.910654 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 07:24:07.910654 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 07:24:07.910654 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 07:24:07.910654 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 07:24:07.911766 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:24:07.911766 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:24:07.911766 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:24:07.911766 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:24:07.911766 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:24:07.911766 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:24:07.911766 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:24:07.911766 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:24:07.911766 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:24:07.911766 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:24:07.911766 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:24:07.911766 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:24:07.911766 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:24:07.911766 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:24:07.911766 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:24:07.911766 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:24:07.911766 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 07:24:07.912820 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 07:24:07.912820 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:24:07.912820 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:24:07.912820 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 623/1224] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:24:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:24:07.918219 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337965c7-65db-4d03-b85d-d6d5ff65254c None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:24:08.027608 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337965c7-65db-4d03-b85d-d6d5ff65254c None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:24:08.061395 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459d515d-a9c8-4b3a-9305-f5270dcf0364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:24:08.065944 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459d515d-a9c8-4b3a-9305-f5270dcf0364 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:08.065944 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459d515d-a9c8-4b3a-9305-f5270dcf0364 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:08.065944 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459d515d-a9c8-4b3a-9305-f5270dcf0364 None admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:08.065944 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459d515d-a9c8-4b3a-9305-f5270dcf0364 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:24:08.065944 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459d515d-a9c8-4b3a-9305-f5270dcf0364 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:24:08.065944 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459d515d-a9c8-4b3a-9305-f5270dcf0364 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:24:08.076988 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 602/1225] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 15 07:24:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:24:08.082818 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c4bdfe-826f-470b-9553-6181499005e4 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:24:08.178188 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:24:08.184959 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c4bdfe-826f-470b-9553-6181499005e4 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:24:08.219145 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8925c348-bbe9-4cc2-84bb-b98214758ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:24:08.219768 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8925c348-bbe9-4cc2-84bb-b98214758ae9 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:08.220047 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8925c348-bbe9-4cc2-84bb-b98214758ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:08.220319 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8925c348-bbe9-4cc2-84bb-b98214758ae9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:08.227551 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8925c348-bbe9-4cc2-84bb-b98214758ae9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:24:08.228982 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8925c348-bbe9-4cc2-84bb-b98214758ae9 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:24:08.240400 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-8925c348-bbe9-4cc2-84bb-b98214758ae9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 15 07:24:08.240400 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:24:08.240400 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:24:08.240400 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:24:08.240400 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:24:08.240400 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:24:08.240400 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:24:08.240400 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:24:08.240400 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:24:08.240400 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:24:08.240400 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:24:08.240400 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:24:08.240400 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 07:24:08.240400 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 07:24:08.240400 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 07:24:08.240400 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 07:24:08.241540 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:24:08.241540 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:24:08.241540 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 07:24:08.241540 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 07:24:08.241540 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 07:24:08.241540 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 07:24:08.241540 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:24:08.241540 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:24:08.241540 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:24:08.241540 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:24:08.241540 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:24:08.241540 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:24:08.241540 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:24:08.241540 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:24:08.241540 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:24:08.241540 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:24:08.241540 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:24:08.241540 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:24:08.242564 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:24:08.242564 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:24:08.242564 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 07:24:08.242564 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 07:24:08.242564 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 07:24:08.242564 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 07:24:08.242564 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 07:24:08.242564 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 07:24:08.242564 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 15 07:24:08.242564 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:24:08.242564 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 624/1226] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:24:08 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:24:08.247451 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459d515d-a9c8-4b3a-9305-f5270dcf0364 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:24:08.295311 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:24:08.347730 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459d515d-a9c8-4b3a-9305-f5270dcf0364 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:24:08.380635 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a808e21e-0520-4935-97e4-df9fcae354b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:24:08.381230 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a808e21e-0520-4935-97e4-df9fcae354b9 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:08.381510 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a808e21e-0520-4935-97e4-df9fcae354b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:08.381810 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a808e21e-0520-4935-97e4-df9fcae354b9 None admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:08.382526 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a808e21e-0520-4935-97e4-df9fcae354b9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:24:08.382830 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a808e21e-0520-4935-97e4-df9fcae354b9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:24:08.384236 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a808e21e-0520-4935-97e4-df9fcae354b9 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:24:08.401963 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 603/1227] 10.222.0.42 () {56 vars in 1143 bytes} [Thu Sep 15 07:24:08 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:24:08.407918 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8925c348-bbe9-4cc2-84bb-b98214758ae9 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:24:08.514635 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8925c348-bbe9-4cc2-84bb-b98214758ae9 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:24:08.547550 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37e31cc-eaf7-4067-ab9f-b13f49104bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:24:08.548113 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37e31cc-eaf7-4067-ab9f-b13f49104bb8 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:08.548391 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37e31cc-eaf7-4067-ab9f-b13f49104bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:08.548703 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37e31cc-eaf7-4067-ab9f-b13f49104bb8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:08.559693 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37e31cc-eaf7-4067-ab9f-b13f49104bb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:24:08.561130 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37e31cc-eaf7-4067-ab9f-b13f49104bb8 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:24:08.575944 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-c37e31cc-eaf7-4067-ab9f-b13f49104bb8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:24:08.575944 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:24:08.575944 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:24:08.575944 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:24:08.575944 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:24:08.575944 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:24:08.575944 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:24:08.575944 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:24:08.575944 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:24:08.575944 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:24:08.575944 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:24:08.575944 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:24:08.575944 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 07:24:08.575944 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 07:24:08.575944 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 07:24:08.575944 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 07:24:08.577068 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:24:08.577068 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:24:08.577068 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:24:08.577068 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:24:08.577068 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:24:08.577068 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:24:08.577068 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:24:08.577068 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:24:08.577068 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:24:08.577068 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:24:08.577068 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:24:08.577068 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:24:08.577068 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:24:08.577068 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:24:08.577068 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:24:08.577068 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:24:08.577068 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 07:24:08.584237 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 07:24:08.584237 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:24:08.584237 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:24:08.584237 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 625/1228] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:24:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:24:08.584555 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a808e21e-0520-4935-97e4-df9fcae354b9 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:24:08.679351 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a808e21e-0520-4935-97e4-df9fcae354b9 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:24:08.681697 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:24:08.714728 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6d1953-b37d-4c56-bbab-80a4182a11f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:24:08.715324 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6d1953-b37d-4c56-bbab-80a4182a11f4 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:08.715604 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6d1953-b37d-4c56-bbab-80a4182a11f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:08.715869 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6d1953-b37d-4c56-bbab-80a4182a11f4 None admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:08.716498 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6d1953-b37d-4c56-bbab-80a4182a11f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:24:08.716797 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6d1953-b37d-4c56-bbab-80a4182a11f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:24:08.718176 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6d1953-b37d-4c56-bbab-80a4182a11f4 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:24:08.728966 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 604/1229] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 15 07:24:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:24:08.734970 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37e31cc-eaf7-4067-ab9f-b13f49104bb8 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:24:08.744224 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:24:08.844050 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37e31cc-eaf7-4067-ab9f-b13f49104bb8 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:24:08.880632 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5314d4c-d50e-4e8f-b1ae-05988deff507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:24:08.881259 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5314d4c-d50e-4e8f-b1ae-05988deff507 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:08.881539 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5314d4c-d50e-4e8f-b1ae-05988deff507 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:08.881843 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5314d4c-d50e-4e8f-b1ae-05988deff507 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:08.888443 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5314d4c-d50e-4e8f-b1ae-05988deff507 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:24:08.889917 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5314d4c-d50e-4e8f-b1ae-05988deff507 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:24:08.900909 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-b5314d4c-d50e-4e8f-b1ae-05988deff507 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 07:24:08.900909 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:24:08.900909 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:24:08.900909 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:24:08.900909 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:24:08.900909 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:24:08.900909 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:24:08.900909 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:24:08.900909 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:24:08.900909 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:24:08.900909 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:24:08.900909 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:24:08.900909 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 07:24:08.900909 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 07:24:08.900909 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 07:24:08.902164 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 07:24:08.902164 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:24:08.902164 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:24:08.902164 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:24:08.902164 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:24:08.902164 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:24:08.902164 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:24:08.902164 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:24:08.902164 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:24:08.902164 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:24:08.902164 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:24:08.902164 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:24:08.902164 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:24:08.902164 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:24:08.902164 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:24:08.902164 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:24:08.902164 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:24:08.903064 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 07:24:08.903064 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 07:24:08.903064 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 07:24:08.903064 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 07:24:08.903064 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 07:24:08.903064 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 07:24:08.903064 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 07:24:08.903064 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:24:08.903064 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 626/1230] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 15 07:24:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:24:08.907908 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6d1953-b37d-4c56-bbab-80a4182a11f4 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:24:09.008660 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6d1953-b37d-4c56-bbab-80a4182a11f4 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:24:09.040746 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811bc454-b35b-4eef-96b1-a40afac349a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:24:09.041346 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811bc454-b35b-4eef-96b1-a40afac349a0 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:09.041652 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811bc454-b35b-4eef-96b1-a40afac349a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:09.042497 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811bc454-b35b-4eef-96b1-a40afac349a0 None admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:09.043269 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811bc454-b35b-4eef-96b1-a40afac349a0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:24:09.043575 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811bc454-b35b-4eef-96b1-a40afac349a0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:24:09.044980 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811bc454-b35b-4eef-96b1-a40afac349a0 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:24:09.056981 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 605/1231] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 15 07:24:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:24:09.063804 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5314d4c-d50e-4e8f-b1ae-05988deff507 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:24:09.129929 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:24:09.164343 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5314d4c-d50e-4e8f-b1ae-05988deff507 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:24:09.198039 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cd4e6d-7e45-43b8-a331-ab1d55885172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:24:09.198717 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cd4e6d-7e45-43b8-a331-ab1d55885172 None admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:09.199038 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cd4e6d-7e45-43b8-a331-ab1d55885172 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:09.199572 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cd4e6d-7e45-43b8-a331-ab1d55885172 None admin] PATH_INFO: `/v3/projects/4e9987720930473da840579654311bc2/users/dd87846b9a134248b28e0521b0a880c9/roles/0e661b5a67664692b933e5da9fc2e488` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:09.213381 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cd4e6d-7e45-43b8-a331-ab1d55885172 None admin] RBAC: Authorizing `identity:create_grant(project_id=4e9987720930473da840579654311bc2, user_id=dd87846b9a134248b28e0521b0a880c9, role_id=0e661b5a67664692b933e5da9fc2e488)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:24:09.215343 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cd4e6d-7e45-43b8-a331-ab1d55885172 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:24:09.285435 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 627/1232] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Sep 15 07:24:09 2022] PUT /identity/v3/projects/4e9987720930473da840579654311bc2/users/dd87846b9a134248b28e0521b0a880c9/roles/0e661b5a67664692b933e5da9fc2e488 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:24:11.432742 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a671b4-343f-43e5-8397-ba3c1e5ca884 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:11.434036 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a671b4-343f-43e5-8397-ba3c1e5ca884 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:11.434036 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a671b4-343f-43e5-8397-ba3c1e5ca884 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:11.435467 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 606/1233] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:24:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:24:11.440763 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b0018f-e3da-460a-a169-c50a6a08a7ae None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:11.441166 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b0018f-e3da-460a-a169-c50a6a08a7ae None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:11.441316 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b0018f-e3da-460a-a169-c50a6a08a7ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:11.459646 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-b6b0018f-e3da-460a-a169-c50a6a08a7ae None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:24:11.460774 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-b6b0018f-e3da-460a-a169-c50a6a08a7ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:24:11.529032 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 628/1234] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:24:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 07:24:11.798010 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a671b4-343f-43e5-8397-ba3c1e5ca884 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:24:11.823379 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:24:11.902795 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a671b4-343f-43e5-8397-ba3c1e5ca884 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:24:11.936050 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc405ec-06e3-47e9-a00c-bd7ca8d00a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:24:11.936664 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc405ec-06e3-47e9-a00c-bd7ca8d00a4e None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:11.936942 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc405ec-06e3-47e9-a00c-bd7ca8d00a4e None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:11.937212 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc405ec-06e3-47e9-a00c-bd7ca8d00a4e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:11.947563 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc405ec-06e3-47e9-a00c-bd7ca8d00a4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:24:11.952841 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-acc405ec-06e3-47e9-a00c-bd7ca8d00a4e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 07:24:11.952841 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:24:11.952841 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:24:11.952841 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:24:11.952841 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:24:11.952841 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:24:11.952841 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:24:11.952841 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:24:11.952841 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:24:11.952841 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:24:11.952841 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:24:11.952841 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:24:11.952841 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:24:11.952841 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:24:11.952841 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:24:11.952841 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:24:11.954079 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:24:11.954079 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:24:11.954079 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:24:11.954079 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:24:11.954079 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:24:11.954079 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:24:11.954079 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:24:11.954079 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:24:11.954079 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:24:11.954079 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:24:11.954079 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:24:11.954079 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:24:11.954079 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:24:11.954079 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:24:11.954079 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:24:11.954079 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:24:11.955129 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:24:11.955129 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:24:11.955129 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:24:11.955129 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:24:11.955129 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:24:11.955129 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:24:11.955129 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:24:11.955129 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:24:11.955129 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:24:11.955129 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:24:11.955129 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 07:24:11.955129 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:24:11.955129 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 607/1235] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Sep 15 07:24:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:24:11.959989 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b0018f-e3da-460a-a169-c50a6a08a7ae None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:24:12.006884 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:24:12.060459 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b0018f-e3da-460a-a169-c50a6a08a7ae None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:24:12.093236 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bad84a9-91e6-4c8c-97dd-91591f378645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:24:12.093876 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bad84a9-91e6-4c8c-97dd-91591f378645 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:12.094153 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bad84a9-91e6-4c8c-97dd-91591f378645 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:12.094427 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bad84a9-91e6-4c8c-97dd-91591f378645 None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:12.095080 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bad84a9-91e6-4c8c-97dd-91591f378645 None admin] RBAC: Adding query filter params (name=service) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:24:12.095377 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bad84a9-91e6-4c8c-97dd-91591f378645 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:24:12.096733 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bad84a9-91e6-4c8c-97dd-91591f378645 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:24:12.104600 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 629/1236] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Sep 15 07:24:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:24:12.111233 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc405ec-06e3-47e9-a00c-bd7ca8d00a4e None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:24:12.197571 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:24:12.213821 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc405ec-06e3-47e9-a00c-bd7ca8d00a4e None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:24:12.254735 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57874a63-e840-4130-a8c8-77ded41627bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:24:12.255373 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57874a63-e840-4130-a8c8-77ded41627bb None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:12.255656 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57874a63-e840-4130-a8c8-77ded41627bb None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:12.255928 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57874a63-e840-4130-a8c8-77ded41627bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:12.267014 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57874a63-e840-4130-a8c8-77ded41627bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:24:12.268447 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57874a63-e840-4130-a8c8-77ded41627bb None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:24:12.282503 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-57874a63-e840-4130-a8c8-77ded41627bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:24:12.282503 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:24:12.282503 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:24:12.282503 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:24:12.282503 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:24:12.282503 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:24:12.282503 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:24:12.282503 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:24:12.282503 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:24:12.282503 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:24:12.282503 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:24:12.282503 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:24:12.282503 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 07:24:12.282503 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 07:24:12.282503 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 07:24:12.282503 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 07:24:12.283641 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:24:12.283641 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:24:12.283641 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:24:12.283641 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:24:12.283641 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:24:12.283641 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:24:12.283641 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:24:12.283641 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:24:12.283641 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:24:12.283641 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:24:12.283641 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:24:12.283641 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:24:12.283641 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:24:12.283641 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:24:12.283641 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:24:12.283641 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:24:12.283641 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 07:24:12.284551 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 07:24:12.284551 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:24:12.284551 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:24:12.284551 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 608/1237] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:24:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:24:12.289407 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bad84a9-91e6-4c8c-97dd-91591f378645 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:24:12.391514 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bad84a9-91e6-4c8c-97dd-91591f378645 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:24:12.405686 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:24:12.436358 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56661fa6-e2d0-4873-a5d5-59911d0fcd89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:24:12.436972 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56661fa6-e2d0-4873-a5d5-59911d0fcd89 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:12.437250 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56661fa6-e2d0-4873-a5d5-59911d0fcd89 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:12.437524 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56661fa6-e2d0-4873-a5d5-59911d0fcd89 None admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:12.438371 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56661fa6-e2d0-4873-a5d5-59911d0fcd89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:24:12.438598 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56661fa6-e2d0-4873-a5d5-59911d0fcd89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:24:12.440045 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56661fa6-e2d0-4873-a5d5-59911d0fcd89 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:24:12.458209 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 630/1238] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 15 07:24:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:24:12.468895 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57874a63-e840-4130-a8c8-77ded41627bb None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:24:12.579955 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57874a63-e840-4130-a8c8-77ded41627bb None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:24:12.616523 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d0b115-fc7b-4568-8cd7-78e606c35b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:24:12.617210 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d0b115-fc7b-4568-8cd7-78e606c35b67 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:12.617489 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d0b115-fc7b-4568-8cd7-78e606c35b67 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:12.617788 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d0b115-fc7b-4568-8cd7-78e606c35b67 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:12.622526 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:24:12.626701 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d0b115-fc7b-4568-8cd7-78e606c35b67 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:24:12.628310 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d0b115-fc7b-4568-8cd7-78e606c35b67 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:24:12.640280 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-d6d0b115-fc7b-4568-8cd7-78e606c35b67 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 15 07:24:12.640280 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:24:12.640280 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:24:12.640280 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:24:12.640280 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:24:12.640280 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:24:12.640280 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:24:12.640280 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:24:12.640280 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:24:12.640280 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:24:12.640280 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:24:12.640280 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:24:12.640280 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 07:24:12.640280 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 07:24:12.640280 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 07:24:12.640280 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 07:24:12.641446 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:24:12.641446 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:24:12.641446 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 07:24:12.641446 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 07:24:12.641446 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 07:24:12.641446 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 07:24:12.641446 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:24:12.641446 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:24:12.641446 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:24:12.641446 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:24:12.641446 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:24:12.641446 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:24:12.641446 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:24:12.641446 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:24:12.641446 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:24:12.641446 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:24:12.641446 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:24:12.641446 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:24:12.642503 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:24:12.642503 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:24:12.642503 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 07:24:12.642503 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 07:24:12.642503 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 07:24:12.642503 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 07:24:12.642503 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 07:24:12.642503 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 07:24:12.642503 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 15 07:24:12.642503 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:24:12.642503 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 609/1239] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:24:12 2022] GET /identity/v3/users/placement => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:24:12.647353 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56661fa6-e2d0-4873-a5d5-59911d0fcd89 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:24:12.771390 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56661fa6-e2d0-4873-a5d5-59911d0fcd89 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:24:12.815745 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccd67d6-0a68-4adf-886e-6b4bc8bb8f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:24:12.816367 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd67d6-0a68-4adf-886e-6b4bc8bb8f5a None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:12.816643 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd67d6-0a68-4adf-886e-6b4bc8bb8f5a None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:12.816913 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd67d6-0a68-4adf-886e-6b4bc8bb8f5a None admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:12.817664 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccd67d6-0a68-4adf-886e-6b4bc8bb8f5a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:24:12.817969 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccd67d6-0a68-4adf-886e-6b4bc8bb8f5a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:24:12.819431 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccd67d6-0a68-4adf-886e-6b4bc8bb8f5a None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:24:12.838236 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 631/1240] 10.222.0.42 () {56 vars in 1143 bytes} [Thu Sep 15 07:24:12 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:24:12.844262 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d0b115-fc7b-4568-8cd7-78e606c35b67 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:24:12.968232 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d0b115-fc7b-4568-8cd7-78e606c35b67 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:24:13.005514 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006c0059-abc7-44ff-9245-e8e0b520c9f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:24:13.006147 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c0059-abc7-44ff-9245-e8e0b520c9f2 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:13.006489 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c0059-abc7-44ff-9245-e8e0b520c9f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:13.006771 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c0059-abc7-44ff-9245-e8e0b520c9f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:13.071903 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:24:13.079083 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006c0059-abc7-44ff-9245-e8e0b520c9f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:24:13.080516 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006c0059-abc7-44ff-9245-e8e0b520c9f2 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:24:13.192648 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 610/1241] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 15 07:24:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 15 07:24:13.203317 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccd67d6-0a68-4adf-886e-6b4bc8bb8f5a None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:24:13.231627 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:24:13.302314 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccd67d6-0a68-4adf-886e-6b4bc8bb8f5a None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:24:13.334995 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cceb25-ceb9-4342-a7cf-49fdf281e49e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:24:13.335664 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cceb25-ceb9-4342-a7cf-49fdf281e49e None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:13.335939 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cceb25-ceb9-4342-a7cf-49fdf281e49e None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:13.336208 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cceb25-ceb9-4342-a7cf-49fdf281e49e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:13.346402 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cceb25-ceb9-4342-a7cf-49fdf281e49e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:24:13.347887 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cceb25-ceb9-4342-a7cf-49fdf281e49e None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:24:13.362586 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-03cceb25-ceb9-4342-a7cf-49fdf281e49e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:24:13.362586 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:24:13.362586 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:24:13.362586 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:24:13.362586 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:24:13.362586 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:24:13.362586 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:24:13.362586 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:24:13.362586 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:24:13.362586 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:24:13.362586 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:24:13.362586 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:24:13.362586 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 07:24:13.362586 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 07:24:13.362586 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 07:24:13.362586 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 07:24:13.363688 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:24:13.363688 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:24:13.363688 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:24:13.363688 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:24:13.363688 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:24:13.363688 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:24:13.363688 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:24:13.363688 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:24:13.363688 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:24:13.363688 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:24:13.363688 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:24:13.363688 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:24:13.363688 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:24:13.363688 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:24:13.363688 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:24:13.363688 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:24:13.363688 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 07:24:13.364816 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 07:24:13.364816 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:24:13.364816 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:24:13.364816 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 632/1242] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:24:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:24:13.371347 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006c0059-abc7-44ff-9245-e8e0b520c9f2 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:24:13.465777 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:24:13.471714 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006c0059-abc7-44ff-9245-e8e0b520c9f2 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:24:13.510587 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf60b27-04d3-4f06-91f0-e27a283633be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:24:13.511233 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf60b27-04d3-4f06-91f0-e27a283633be None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:13.511566 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf60b27-04d3-4f06-91f0-e27a283633be None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:13.511926 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf60b27-04d3-4f06-91f0-e27a283633be None admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:13.512601 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf60b27-04d3-4f06-91f0-e27a283633be None admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:24:13.512897 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf60b27-04d3-4f06-91f0-e27a283633be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:24:13.514265 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf60b27-04d3-4f06-91f0-e27a283633be None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:24:13.525182 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 611/1243] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 15 07:24:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:24:13.531391 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cceb25-ceb9-4342-a7cf-49fdf281e49e None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:24:13.611907 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:24:13.633059 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cceb25-ceb9-4342-a7cf-49fdf281e49e None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:24:13.664799 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65eb79a-4a72-496c-8d94-d74847d88ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:24:13.665421 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65eb79a-4a72-496c-8d94-d74847d88ed8 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:13.665724 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65eb79a-4a72-496c-8d94-d74847d88ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:13.665995 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65eb79a-4a72-496c-8d94-d74847d88ed8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:13.672929 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65eb79a-4a72-496c-8d94-d74847d88ed8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:24:13.674396 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65eb79a-4a72-496c-8d94-d74847d88ed8 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:24:13.685301 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-a65eb79a-4a72-496c-8d94-d74847d88ed8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 07:24:13.685301 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:24:13.685301 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:24:13.685301 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:24:13.685301 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:24:13.685301 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:24:13.685301 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:24:13.685301 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:24:13.685301 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:24:13.685301 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:24:13.685301 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:24:13.685301 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:24:13.685301 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 07:24:13.685301 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 07:24:13.685301 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 07:24:13.686255 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 07:24:13.686255 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:24:13.686255 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:24:13.686255 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:24:13.686255 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:24:13.686255 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:24:13.686255 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:24:13.686255 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:24:13.686255 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:24:13.686255 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:24:13.686255 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:24:13.686255 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:24:13.686255 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:24:13.686255 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:24:13.686255 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:24:13.686255 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:24:13.686255 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:24:13.687335 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 07:24:13.687335 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 07:24:13.687335 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 07:24:13.687335 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 07:24:13.687335 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 07:24:13.687335 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 07:24:13.687335 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 07:24:13.687335 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:24:13.687335 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 633/1244] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 15 07:24:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:24:13.692252 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf60b27-04d3-4f06-91f0-e27a283633be None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:24:13.791183 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf60b27-04d3-4f06-91f0-e27a283633be None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:24:13.823818 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70a8a48-4d0d-4b75-b788-720fb45f5df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:24:13.824422 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70a8a48-4d0d-4b75-b788-720fb45f5df2 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:13.824960 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70a8a48-4d0d-4b75-b788-720fb45f5df2 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:13.825231 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70a8a48-4d0d-4b75-b788-720fb45f5df2 None admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:13.825945 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70a8a48-4d0d-4b75-b788-720fb45f5df2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:24:13.826247 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70a8a48-4d0d-4b75-b788-720fb45f5df2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:24:13.827718 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70a8a48-4d0d-4b75-b788-720fb45f5df2 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:24:13.839969 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 612/1245] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 15 07:24:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:24:13.846572 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65eb79a-4a72-496c-8d94-d74847d88ed8 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:24:13.947364 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65eb79a-4a72-496c-8d94-d74847d88ed8 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:24:13.983990 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a935acf1-db3c-4a8c-bad0-f29839db1b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:24:13.984599 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a935acf1-db3c-4a8c-bad0-f29839db1b2e None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:13.984945 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a935acf1-db3c-4a8c-bad0-f29839db1b2e None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:13.985184 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a935acf1-db3c-4a8c-bad0-f29839db1b2e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:13.985927 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a935acf1-db3c-4a8c-bad0-f29839db1b2e None admin] RBAC: Adding query filter params (role.id=0e661b5a67664692b933e5da9fc2e488, scope.project.id=4e9987720930473da840579654311bc2, user.id=dd87846b9a134248b28e0521b0a880c9) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:24:13.986237 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a935acf1-db3c-4a8c-bad0-f29839db1b2e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:24:13.987653 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a935acf1-db3c-4a8c-bad0-f29839db1b2e None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:24:13.993338 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 634/1246] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 15 07:24:13 2022] GET /identity/v3/role_assignments?user.id=dd87846b9a134248b28e0521b0a880c9&scope.project.id=4e9987720930473da840579654311bc2&role.id=0e661b5a67664692b933e5da9fc2e488 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:24:16.232518 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f37c7a-ec3d-4630-aa96-383372285bb6 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:16.233013 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f37c7a-ec3d-4630-aa96-383372285bb6 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:16.233013 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f37c7a-ec3d-4630-aa96-383372285bb6 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:16.234382 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 613/1247] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:24:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:24:16.239762 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a756460a-481e-4f6d-8328-6ea4554338af None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:16.240178 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a756460a-481e-4f6d-8328-6ea4554338af None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:16.240288 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a756460a-481e-4f6d-8328-6ea4554338af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:16.247287 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:24:16.259451 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-a756460a-481e-4f6d-8328-6ea4554338af None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:24:16.260634 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-a756460a-481e-4f6d-8328-6ea4554338af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:24:16.326559 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 635/1248] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:24:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 07:24:16.601692 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f37c7a-ec3d-4630-aa96-383372285bb6 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:24:16.620466 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:24:16.704443 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f37c7a-ec3d-4630-aa96-383372285bb6 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:24:16.738940 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db95a972-fe70-4502-8670-927bd8b47c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:24:16.739563 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db95a972-fe70-4502-8670-927bd8b47c06 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:16.739844 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db95a972-fe70-4502-8670-927bd8b47c06 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:16.740116 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db95a972-fe70-4502-8670-927bd8b47c06 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:16.750495 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db95a972-fe70-4502-8670-927bd8b47c06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:24:16.755303 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-db95a972-fe70-4502-8670-927bd8b47c06 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 07:24:16.755303 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:24:16.755303 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:24:16.755303 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:24:16.755303 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:24:16.755303 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:24:16.755303 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:24:16.755303 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:24:16.755303 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:24:16.755303 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:24:16.755303 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:24:16.755303 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:24:16.755303 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:24:16.755303 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:24:16.755303 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:24:16.755303 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:24:16.756654 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:24:16.756654 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:24:16.756654 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:24:16.756654 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:24:16.756654 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:24:16.756654 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:24:16.756654 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:24:16.756654 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:24:16.756654 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:24:16.756654 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:24:16.756654 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:24:16.756654 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:24:16.756654 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:24:16.756654 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:24:16.756654 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:24:16.756654 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:24:16.757566 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:24:16.757566 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:24:16.757566 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:24:16.757566 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:24:16.757566 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:24:16.757566 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:24:16.757566 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:24:16.757566 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:24:16.757566 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:24:16.757566 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:24:16.757566 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 07:24:16.757566 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:24:16.762010 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 614/1249] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 15 07:24:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:24:16.762882 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a756460a-481e-4f6d-8328-6ea4554338af None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:24:16.876598 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a756460a-481e-4f6d-8328-6ea4554338af None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:24:16.906514 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:24:16.911502 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a429311-646e-4877-a596-acd580f9a440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:24:16.912117 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a429311-646e-4877-a596-acd580f9a440 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:16.912399 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a429311-646e-4877-a596-acd580f9a440 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:16.912677 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a429311-646e-4877-a596-acd580f9a440 None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:16.913339 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a429311-646e-4877-a596-acd580f9a440 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:24:16.913682 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a429311-646e-4877-a596-acd580f9a440 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:24:16.915044 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a429311-646e-4877-a596-acd580f9a440 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:24:16.923441 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 636/1250] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 15 07:24:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:24:16.929358 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db95a972-fe70-4502-8670-927bd8b47c06 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:24:17.008140 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:24:17.031806 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db95a972-fe70-4502-8670-927bd8b47c06 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:24:17.065028 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a2d6a0-5c6f-43a8-aeee-85d15a650b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:24:17.065753 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a2d6a0-5c6f-43a8-aeee-85d15a650b6e None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:17.066039 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a2d6a0-5c6f-43a8-aeee-85d15a650b6e None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:17.066316 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a2d6a0-5c6f-43a8-aeee-85d15a650b6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:17.077864 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a2d6a0-5c6f-43a8-aeee-85d15a650b6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:24:17.079346 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a2d6a0-5c6f-43a8-aeee-85d15a650b6e None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:24:17.093811 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-17a2d6a0-5c6f-43a8-aeee-85d15a650b6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:24:17.093811 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:24:17.093811 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:24:17.093811 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:24:17.093811 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:24:17.093811 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:24:17.093811 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:24:17.093811 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:24:17.093811 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:24:17.093811 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:24:17.093811 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:24:17.093811 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:24:17.093811 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 07:24:17.093811 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 07:24:17.093811 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 07:24:17.093811 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 07:24:17.094921 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:24:17.094921 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:24:17.094921 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:24:17.094921 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:24:17.094921 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:24:17.094921 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:24:17.094921 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:24:17.094921 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:24:17.094921 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:24:17.094921 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:24:17.094921 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:24:17.094921 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:24:17.094921 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:24:17.094921 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:24:17.094921 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:24:17.094921 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:24:17.094921 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 07:24:17.096174 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 07:24:17.096174 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:24:17.096174 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:24:17.096174 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 615/1251] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:24:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:24:17.100674 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a429311-646e-4877-a596-acd580f9a440 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:24:17.200737 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a429311-646e-4877-a596-acd580f9a440 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:24:17.233986 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8d5b43-5bb9-4856-9a9a-fe0ce52e3523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:24:17.234594 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8d5b43-5bb9-4856-9a9a-fe0ce52e3523 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:17.234875 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8d5b43-5bb9-4856-9a9a-fe0ce52e3523 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:17.235149 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8d5b43-5bb9-4856-9a9a-fe0ce52e3523 None admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:17.235794 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8d5b43-5bb9-4856-9a9a-fe0ce52e3523 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:24:17.236102 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8d5b43-5bb9-4856-9a9a-fe0ce52e3523 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:24:17.237471 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8d5b43-5bb9-4856-9a9a-fe0ce52e3523 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:24:17.249857 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 637/1252] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 15 07:24:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:24:17.255694 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a2d6a0-5c6f-43a8-aeee-85d15a650b6e None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:24:17.353637 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a2d6a0-5c6f-43a8-aeee-85d15a650b6e None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:24:17.380082 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:24:17.387916 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e7d831-f619-4252-909b-7c5ce1416eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:24:17.388557 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e7d831-f619-4252-909b-7c5ce1416eaa None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:17.388841 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e7d831-f619-4252-909b-7c5ce1416eaa None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:17.389120 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e7d831-f619-4252-909b-7c5ce1416eaa None admin] PATH_INFO: `/v3/users/placement` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:17.397415 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e7d831-f619-4252-909b-7c5ce1416eaa None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:24:17.398883 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e7d831-f619-4252-909b-7c5ce1416eaa None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:24:17.409889 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-54e7d831-f619-4252-909b-7c5ce1416eaa None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 15 07:24:17.409889 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:24:17.409889 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:24:17.409889 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:24:17.409889 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:24:17.409889 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:24:17.409889 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:24:17.409889 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:24:17.409889 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:24:17.409889 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:24:17.409889 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:24:17.409889 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:24:17.409889 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 07:24:17.409889 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 07:24:17.409889 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 07:24:17.409889 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 07:24:17.411118 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:24:17.411118 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:24:17.411118 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 07:24:17.411118 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 07:24:17.411118 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 07:24:17.411118 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 07:24:17.411118 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:24:17.411118 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:24:17.411118 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:24:17.411118 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:24:17.411118 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:24:17.411118 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:24:17.411118 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:24:17.411118 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:24:17.411118 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:24:17.411118 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:24:17.411118 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:24:17.411118 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:24:17.412052 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:24:17.412052 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:24:17.412052 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 07:24:17.412052 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 07:24:17.412052 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 07:24:17.412052 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 07:24:17.412052 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 07:24:17.412052 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 07:24:17.412052 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 15 07:24:17.412052 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:24:17.412052 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 616/1253] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:24:17 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:24:17.416961 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8d5b43-5bb9-4856-9a9a-fe0ce52e3523 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:24:17.473330 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:24:17.516433 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8d5b43-5bb9-4856-9a9a-fe0ce52e3523 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:24:17.549806 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55cdf2d-bdcf-498e-9d21-6ae8b46ce7fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:24:17.550399 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55cdf2d-bdcf-498e-9d21-6ae8b46ce7fd None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:17.550683 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55cdf2d-bdcf-498e-9d21-6ae8b46ce7fd None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:17.550959 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55cdf2d-bdcf-498e-9d21-6ae8b46ce7fd None admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:17.551676 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55cdf2d-bdcf-498e-9d21-6ae8b46ce7fd None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:24:17.551985 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55cdf2d-bdcf-498e-9d21-6ae8b46ce7fd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:24:17.553390 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55cdf2d-bdcf-498e-9d21-6ae8b46ce7fd None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:24:17.570003 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 638/1254] 10.222.0.42 () {56 vars in 1143 bytes} [Thu Sep 15 07:24:17 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:24:17.576060 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e7d831-f619-4252-909b-7c5ce1416eaa None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:24:17.675033 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e7d831-f619-4252-909b-7c5ce1416eaa None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:24:17.708452 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f851cafa-3d94-4ffc-b61b-0c3ff45fc969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:24:17.709057 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f851cafa-3d94-4ffc-b61b-0c3ff45fc969 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:17.709338 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f851cafa-3d94-4ffc-b61b-0c3ff45fc969 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:17.709642 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f851cafa-3d94-4ffc-b61b-0c3ff45fc969 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:17.764711 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:24:17.776485 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f851cafa-3d94-4ffc-b61b-0c3ff45fc969 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:24:17.778803 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f851cafa-3d94-4ffc-b61b-0c3ff45fc969 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:24:17.875704 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 617/1255] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 15 07:24:17 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 07:24:17.881541 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55cdf2d-bdcf-498e-9d21-6ae8b46ce7fd None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:24:17.985327 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55cdf2d-bdcf-498e-9d21-6ae8b46ce7fd None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:24:17.998281 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:24:18.018954 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669c594d-96ab-40af-ad41-da6a4e44dc29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:24:18.019510 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669c594d-96ab-40af-ad41-da6a4e44dc29 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:18.019787 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669c594d-96ab-40af-ad41-da6a4e44dc29 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:18.020055 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669c594d-96ab-40af-ad41-da6a4e44dc29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:18.030618 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669c594d-96ab-40af-ad41-da6a4e44dc29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:24:18.032061 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669c594d-96ab-40af-ad41-da6a4e44dc29 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:24:18.046578 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-669c594d-96ab-40af-ad41-da6a4e44dc29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:24:18.046578 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:24:18.046578 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:24:18.046578 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:24:18.046578 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:24:18.046578 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:24:18.046578 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:24:18.046578 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:24:18.046578 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:24:18.046578 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:24:18.046578 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:24:18.046578 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:24:18.046578 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 07:24:18.046578 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 07:24:18.046578 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 07:24:18.046578 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 07:24:18.047779 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:24:18.047779 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:24:18.047779 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:24:18.047779 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:24:18.047779 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:24:18.047779 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:24:18.047779 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:24:18.047779 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:24:18.047779 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:24:18.047779 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:24:18.047779 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:24:18.047779 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:24:18.047779 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:24:18.047779 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:24:18.047779 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:24:18.047779 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:24:18.047779 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 07:24:18.048929 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 07:24:18.048929 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:24:18.048929 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:24:18.048929 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 639/1256] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:24:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:24:18.053717 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f851cafa-3d94-4ffc-b61b-0c3ff45fc969 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:24:18.161471 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:24:18.173333 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f851cafa-3d94-4ffc-b61b-0c3ff45fc969 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:24:18.207693 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fa74bd-d075-4db1-8941-3878b2577c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:24:18.208301 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fa74bd-d075-4db1-8941-3878b2577c29 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:18.208585 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fa74bd-d075-4db1-8941-3878b2577c29 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:18.208862 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fa74bd-d075-4db1-8941-3878b2577c29 None admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:18.209500 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fa74bd-d075-4db1-8941-3878b2577c29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:24:18.209863 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fa74bd-d075-4db1-8941-3878b2577c29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:24:18.211209 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fa74bd-d075-4db1-8941-3878b2577c29 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:24:18.223081 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 618/1257] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 15 07:24:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:24:18.229006 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669c594d-96ab-40af-ad41-da6a4e44dc29 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:24:18.360999 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669c594d-96ab-40af-ad41-da6a4e44dc29 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:24:18.399601 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3b8d1f-3d8e-40f7-b63c-8391d14e820b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:24:18.400229 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3b8d1f-3d8e-40f7-b63c-8391d14e820b None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:18.400509 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3b8d1f-3d8e-40f7-b63c-8391d14e820b None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:18.400781 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3b8d1f-3d8e-40f7-b63c-8391d14e820b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:18.407482 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3b8d1f-3d8e-40f7-b63c-8391d14e820b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:24:18.408922 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3b8d1f-3d8e-40f7-b63c-8391d14e820b None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:24:18.420224 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-da3b8d1f-3d8e-40f7-b63c-8391d14e820b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 07:24:18.420224 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:24:18.420224 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:24:18.420224 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:24:18.420224 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:24:18.420224 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:24:18.420224 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:24:18.420224 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:24:18.420224 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:24:18.420224 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:24:18.420224 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:24:18.420224 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:24:18.420224 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 07:24:18.420224 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 07:24:18.420224 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 07:24:18.421227 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 07:24:18.421227 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:24:18.421227 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:24:18.421227 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:24:18.421227 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:24:18.421227 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:24:18.421227 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:24:18.421227 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:24:18.421227 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:24:18.421227 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:24:18.421227 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:24:18.421227 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:24:18.421227 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:24:18.421227 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:24:18.421227 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:24:18.421227 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:24:18.421227 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:24:18.422211 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 07:24:18.422211 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 07:24:18.422211 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 07:24:18.422211 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 07:24:18.422211 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 07:24:18.422211 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 07:24:18.422211 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 07:24:18.422211 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:24:18.422211 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 640/1258] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 15 07:24:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:24:18.427553 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fa74bd-d075-4db1-8941-3878b2577c29 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:24:18.528716 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fa74bd-d075-4db1-8941-3878b2577c29 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:24:18.561806 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50731ae2-ebe7-4008-88b8-35091b30e559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:24:18.562404 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50731ae2-ebe7-4008-88b8-35091b30e559 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:18.562682 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50731ae2-ebe7-4008-88b8-35091b30e559 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:18.562952 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50731ae2-ebe7-4008-88b8-35091b30e559 None admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:18.563632 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50731ae2-ebe7-4008-88b8-35091b30e559 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:24:18.563933 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50731ae2-ebe7-4008-88b8-35091b30e559 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:24:18.565412 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50731ae2-ebe7-4008-88b8-35091b30e559 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:24:18.577466 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 619/1259] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 15 07:24:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:24:18.583739 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3b8d1f-3d8e-40f7-b63c-8391d14e820b None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:24:18.611279 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:24:18.685043 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3b8d1f-3d8e-40f7-b63c-8391d14e820b None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:24:18.719500 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f59d518-452f-4047-9289-f3c9e1e46699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:24:18.720118 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f59d518-452f-4047-9289-f3c9e1e46699 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:18.720396 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f59d518-452f-4047-9289-f3c9e1e46699 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:18.720678 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f59d518-452f-4047-9289-f3c9e1e46699 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:18.721441 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f59d518-452f-4047-9289-f3c9e1e46699 None admin] RBAC: Adding query filter params (role.id=6bbb643e0d8d4bc0a9b27c26edf59216, scope.project.id=4e9987720930473da840579654311bc2, user.id=dd87846b9a134248b28e0521b0a880c9) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:24:18.721784 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f59d518-452f-4047-9289-f3c9e1e46699 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:24:18.723213 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f59d518-452f-4047-9289-f3c9e1e46699 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:24:18.729112 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 641/1260] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 15 07:24:18 2022] GET /identity/v3/role_assignments?user.id=dd87846b9a134248b28e0521b0a880c9&scope.project.id=4e9987720930473da840579654311bc2&role.id=6bbb643e0d8d4bc0a9b27c26edf59216 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:24:20.858980 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278bf324-77fc-4f05-80bb-b74d34def20f None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:20.859664 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278bf324-77fc-4f05-80bb-b74d34def20f None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:20.859664 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278bf324-77fc-4f05-80bb-b74d34def20f None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:20.862081 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 620/1261] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:24:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:24:20.868537 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29c407b-d8bd-422b-a56b-0ee3dec55b25 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:20.869049 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29c407b-d8bd-422b-a56b-0ee3dec55b25 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:20.869552 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29c407b-d8bd-422b-a56b-0ee3dec55b25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:20.888687 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-f29c407b-d8bd-422b-a56b-0ee3dec55b25 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:24:20.889929 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-f29c407b-d8bd-422b-a56b-0ee3dec55b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:24:20.958210 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 642/1262] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:24:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 07:24:21.227228 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278bf324-77fc-4f05-80bb-b74d34def20f None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:24:21.238674 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:24:21.327025 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278bf324-77fc-4f05-80bb-b74d34def20f None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:24:21.363170 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e671fa6-8dff-4ced-b402-3c8012cd4f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:24:21.363800 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e671fa6-8dff-4ced-b402-3c8012cd4f86 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:21.364075 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e671fa6-8dff-4ced-b402-3c8012cd4f86 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:21.364342 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e671fa6-8dff-4ced-b402-3c8012cd4f86 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:21.374967 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e671fa6-8dff-4ced-b402-3c8012cd4f86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:24:21.379898 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-1e671fa6-8dff-4ced-b402-3c8012cd4f86 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 07:24:21.379898 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:24:21.379898 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:24:21.379898 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:24:21.379898 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:24:21.379898 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:24:21.379898 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:24:21.379898 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:24:21.379898 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:24:21.379898 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:24:21.379898 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:24:21.379898 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:24:21.379898 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:24:21.379898 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:24:21.379898 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:24:21.379898 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:24:21.385666 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:24:21.385666 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:24:21.385666 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:24:21.385666 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:24:21.385666 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:24:21.385666 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:24:21.385666 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:24:21.385666 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:24:21.385666 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:24:21.385666 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:24:21.385666 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:24:21.385666 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:24:21.385666 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:24:21.385666 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:24:21.385666 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:24:21.385666 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:24:21.386698 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:24:21.386698 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:24:21.386698 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:24:21.386698 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:24:21.386698 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:24:21.386698 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:24:21.386698 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:24:21.386698 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:24:21.386698 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:24:21.386698 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:24:21.386698 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 07:24:21.386698 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:24:21.386698 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 621/1263] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 15 07:24:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:24:21.387564 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29c407b-d8bd-422b-a56b-0ee3dec55b25 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:24:21.406001 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:24:21.485090 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29c407b-d8bd-422b-a56b-0ee3dec55b25 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:24:21.517527 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c4f726-a109-4425-9eec-a2f1f4bd84b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:24:21.518152 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c4f726-a109-4425-9eec-a2f1f4bd84b1 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:21.518435 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c4f726-a109-4425-9eec-a2f1f4bd84b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:21.518706 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c4f726-a109-4425-9eec-a2f1f4bd84b1 None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:21.519358 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c4f726-a109-4425-9eec-a2f1f4bd84b1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:24:21.519663 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c4f726-a109-4425-9eec-a2f1f4bd84b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:24:21.521023 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c4f726-a109-4425-9eec-a2f1f4bd84b1 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:24:21.529711 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 643/1264] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 15 07:24:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:24:21.535403 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e671fa6-8dff-4ced-b402-3c8012cd4f86 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:24:21.601027 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:24:21.634350 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e671fa6-8dff-4ced-b402-3c8012cd4f86 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:24:21.669747 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff44c3e-dd10-43e2-8e14-76213436b0dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:24:21.670370 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff44c3e-dd10-43e2-8e14-76213436b0dc None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:21.670673 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff44c3e-dd10-43e2-8e14-76213436b0dc None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:21.670971 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff44c3e-dd10-43e2-8e14-76213436b0dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:21.681372 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff44c3e-dd10-43e2-8e14-76213436b0dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:24:21.682906 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff44c3e-dd10-43e2-8e14-76213436b0dc None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:24:21.696884 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-6ff44c3e-dd10-43e2-8e14-76213436b0dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:24:21.696884 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:24:21.696884 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:24:21.696884 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:24:21.696884 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:24:21.696884 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:24:21.696884 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:24:21.696884 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:24:21.696884 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:24:21.696884 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:24:21.696884 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:24:21.696884 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:24:21.696884 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 07:24:21.696884 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 07:24:21.696884 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 07:24:21.696884 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 07:24:21.698044 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:24:21.698044 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:24:21.698044 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:24:21.698044 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:24:21.698044 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:24:21.698044 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:24:21.698044 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:24:21.698044 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:24:21.698044 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:24:21.698044 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:24:21.698044 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:24:21.698044 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:24:21.698044 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:24:21.698044 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:24:21.698044 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:24:21.698044 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:24:21.698044 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 07:24:21.699794 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 07:24:21.699794 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:24:21.699794 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:24:21.699794 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 622/1265] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:24:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:24:21.704586 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c4f726-a109-4425-9eec-a2f1f4bd84b1 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:24:21.786814 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:24:21.805128 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c4f726-a109-4425-9eec-a2f1f4bd84b1 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:24:21.838556 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a6ce90-a768-4562-8e09-004955171c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:24:21.839157 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a6ce90-a768-4562-8e09-004955171c0a None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:21.839436 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a6ce90-a768-4562-8e09-004955171c0a None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:21.840296 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a6ce90-a768-4562-8e09-004955171c0a None admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:21.841091 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a6ce90-a768-4562-8e09-004955171c0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:24:21.841398 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a6ce90-a768-4562-8e09-004955171c0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:24:21.842787 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a6ce90-a768-4562-8e09-004955171c0a None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:24:21.857843 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 644/1266] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 15 07:24:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:24:21.861509 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff44c3e-dd10-43e2-8e14-76213436b0dc None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:24:21.961838 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff44c3e-dd10-43e2-8e14-76213436b0dc None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:24:21.982773 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:24:21.997097 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e60578-f13e-4946-af53-b06cfec84dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:24:21.997740 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e60578-f13e-4946-af53-b06cfec84dd5 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:21.998024 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e60578-f13e-4946-af53-b06cfec84dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:21.998296 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e60578-f13e-4946-af53-b06cfec84dd5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:22.005689 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e60578-f13e-4946-af53-b06cfec84dd5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:24:22.007123 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e60578-f13e-4946-af53-b06cfec84dd5 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:24:22.018862 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-07e60578-f13e-4946-af53-b06cfec84dd5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 15 07:24:22.018862 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:24:22.018862 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:24:22.018862 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:24:22.018862 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:24:22.018862 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:24:22.018862 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:24:22.018862 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:24:22.018862 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:24:22.018862 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:24:22.018862 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:24:22.018862 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:24:22.018862 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 07:24:22.018862 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 07:24:22.018862 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 07:24:22.018862 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 07:24:22.019997 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:24:22.019997 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:24:22.019997 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 07:24:22.019997 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 07:24:22.019997 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 07:24:22.019997 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 07:24:22.019997 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:24:22.019997 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:24:22.019997 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:24:22.019997 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:24:22.019997 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:24:22.019997 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:24:22.019997 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:24:22.019997 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:24:22.019997 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:24:22.019997 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:24:22.019997 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:24:22.019997 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:24:22.021166 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:24:22.021166 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:24:22.021166 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 07:24:22.021166 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 07:24:22.021166 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 07:24:22.021166 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 07:24:22.021166 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 07:24:22.021166 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 07:24:22.021166 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 15 07:24:22.021166 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:24:22.021166 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 623/1267] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:24:21 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:24:22.025953 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a6ce90-a768-4562-8e09-004955171c0a None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:24:22.140584 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a6ce90-a768-4562-8e09-004955171c0a None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:24:22.173745 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5567a0cd-224a-47b9-bec7-1d43741e3b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:24:22.174385 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5567a0cd-224a-47b9-bec7-1d43741e3b22 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:22.174660 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5567a0cd-224a-47b9-bec7-1d43741e3b22 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:22.174929 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5567a0cd-224a-47b9-bec7-1d43741e3b22 None admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:22.175637 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5567a0cd-224a-47b9-bec7-1d43741e3b22 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:24:22.175937 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5567a0cd-224a-47b9-bec7-1d43741e3b22 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:24:22.177415 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5567a0cd-224a-47b9-bec7-1d43741e3b22 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:24:22.186539 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:24:22.194990 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 645/1268] 10.222.0.42 () {56 vars in 1143 bytes} [Thu Sep 15 07:24:22 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:24:22.201844 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e60578-f13e-4946-af53-b06cfec84dd5 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:24:22.299332 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e60578-f13e-4946-af53-b06cfec84dd5 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:24:22.332664 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddf998d-4393-4e61-97e6-827d5fafcbb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:24:22.333276 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddf998d-4393-4e61-97e6-827d5fafcbb2 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:22.333559 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddf998d-4393-4e61-97e6-827d5fafcbb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:22.333888 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddf998d-4393-4e61-97e6-827d5fafcbb2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:22.344093 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddf998d-4393-4e61-97e6-827d5fafcbb2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:24:22.345508 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddf998d-4393-4e61-97e6-827d5fafcbb2 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:24:22.360282 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-cddf998d-4393-4e61-97e6-827d5fafcbb2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:24:22.360282 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:24:22.360282 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:24:22.360282 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:24:22.360282 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:24:22.360282 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:24:22.360282 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:24:22.360282 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:24:22.360282 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:24:22.360282 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:24:22.360282 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:24:22.360282 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:24:22.360282 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 07:24:22.360282 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 07:24:22.360282 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 07:24:22.360282 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 07:24:22.361429 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:24:22.361429 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:24:22.361429 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:24:22.361429 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:24:22.361429 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:24:22.361429 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:24:22.361429 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:24:22.361429 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:24:22.361429 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:24:22.361429 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:24:22.361429 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:24:22.361429 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:24:22.361429 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:24:22.361429 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:24:22.361429 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:24:22.361429 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:24:22.361429 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 07:24:22.362380 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 07:24:22.362380 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:24:22.362380 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:24:22.362380 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 624/1269] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:24:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:24:22.367092 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5567a0cd-224a-47b9-bec7-1d43741e3b22 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:24:22.472008 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5567a0cd-224a-47b9-bec7-1d43741e3b22 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:24:22.506732 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2726641-cf4d-4099-8a8a-e1bb425652f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:24:22.507330 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2726641-cf4d-4099-8a8a-e1bb425652f9 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:22.507619 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2726641-cf4d-4099-8a8a-e1bb425652f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:22.507892 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2726641-cf4d-4099-8a8a-e1bb425652f9 None admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:22.508524 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2726641-cf4d-4099-8a8a-e1bb425652f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:24:22.508828 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2726641-cf4d-4099-8a8a-e1bb425652f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:24:22.510217 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2726641-cf4d-4099-8a8a-e1bb425652f9 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:24:22.521523 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 646/1270] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 15 07:24:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:24:22.527522 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddf998d-4393-4e61-97e6-827d5fafcbb2 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:24:22.561073 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:24:22.628150 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddf998d-4393-4e61-97e6-827d5fafcbb2 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:24:22.662746 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddf18f6-3349-4be4-b2d1-3ddf53e83a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:24:22.663364 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddf18f6-3349-4be4-b2d1-3ddf53e83a13 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:22.663640 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddf18f6-3349-4be4-b2d1-3ddf53e83a13 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:22.663910 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddf18f6-3349-4be4-b2d1-3ddf53e83a13 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:22.671205 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddf18f6-3349-4be4-b2d1-3ddf53e83a13 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:24:22.672610 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddf18f6-3349-4be4-b2d1-3ddf53e83a13 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:24:22.684341 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-2ddf18f6-3349-4be4-b2d1-3ddf53e83a13 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 07:24:22.684341 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:24:22.684341 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:24:22.684341 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:24:22.684341 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:24:22.684341 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:24:22.684341 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:24:22.684341 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:24:22.684341 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:24:22.684341 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:24:22.684341 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:24:22.684341 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:24:22.684341 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 07:24:22.684341 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 07:24:22.684341 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 07:24:22.690249 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 07:24:22.690249 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:24:22.690249 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:24:22.690249 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:24:22.690249 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:24:22.690249 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:24:22.690249 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:24:22.690249 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:24:22.690249 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:24:22.690249 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:24:22.690249 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:24:22.690249 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:24:22.690249 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:24:22.690249 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:24:22.690249 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:24:22.690249 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:24:22.690249 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:24:22.691349 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 07:24:22.691349 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 07:24:22.691349 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 07:24:22.691349 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 07:24:22.691349 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 07:24:22.691349 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 07:24:22.691349 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 07:24:22.691349 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:24:22.691349 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 625/1271] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 15 07:24:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:24:22.691943 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2726641-cf4d-4099-8a8a-e1bb425652f9 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:24:22.751281 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:24:22.792759 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2726641-cf4d-4099-8a8a-e1bb425652f9 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:24:22.826829 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d9a72c-4fb6-49c5-88f8-f3b5c0633de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:24:22.827436 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d9a72c-4fb6-49c5-88f8-f3b5c0633de5 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:22.827717 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d9a72c-4fb6-49c5-88f8-f3b5c0633de5 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:22.827989 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d9a72c-4fb6-49c5-88f8-f3b5c0633de5 None admin] PATH_INFO: `/v3/projects` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:22.828677 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d9a72c-4fb6-49c5-88f8-f3b5c0633de5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:24:22.828978 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d9a72c-4fb6-49c5-88f8-f3b5c0633de5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:24:22.830426 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d9a72c-4fb6-49c5-88f8-f3b5c0633de5 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:24:22.844220 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 647/1272] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 15 07:24:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:24:22.850403 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddf18f6-3349-4be4-b2d1-3ddf53e83a13 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:24:22.944898 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:24:22.954434 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddf18f6-3349-4be4-b2d1-3ddf53e83a13 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:24:22.991869 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f033a71d-dc45-4272-bb7f-46d8ef549a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:24:22.992544 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f033a71d-dc45-4272-bb7f-46d8ef549a98 None admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:22.992828 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f033a71d-dc45-4272-bb7f-46d8ef549a98 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:22.993105 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f033a71d-dc45-4272-bb7f-46d8ef549a98 None admin] PATH_INFO: `/v3/projects/4e9987720930473da840579654311bc2/users/dd87846b9a134248b28e0521b0a880c9/roles/6bbb643e0d8d4bc0a9b27c26edf59216` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:22.995108 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f033a71d-dc45-4272-bb7f-46d8ef549a98 None admin] RBAC: Authorizing `identity:create_grant(project_id=4e9987720930473da840579654311bc2, user_id=dd87846b9a134248b28e0521b0a880c9, role_id=6bbb643e0d8d4bc0a9b27c26edf59216)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:24:22.996771 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f033a71d-dc45-4272-bb7f-46d8ef549a98 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:24:23.060944 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 626/1273] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Sep 15 07:24:22 2022] PUT /identity/v3/projects/4e9987720930473da840579654311bc2/users/dd87846b9a134248b28e0521b0a880c9/roles/6bbb643e0d8d4bc0a9b27c26edf59216 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:24:25.195264 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea424915-8baf-4505-a5c8-2320f9bb50d8 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:25.195802 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea424915-8baf-4505-a5c8-2320f9bb50d8 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:25.195802 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea424915-8baf-4505-a5c8-2320f9bb50d8 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:25.197352 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 648/1274] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:24:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:24:25.204388 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f779d109-82d5-4fc1-a8f4-13bf6c0c281e None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:25.207191 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f779d109-82d5-4fc1-a8f4-13bf6c0c281e None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:25.207191 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f779d109-82d5-4fc1-a8f4-13bf6c0c281e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:25.224159 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-f779d109-82d5-4fc1-a8f4-13bf6c0c281e None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:24:25.225359 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-f779d109-82d5-4fc1-a8f4-13bf6c0c281e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:24:25.292769 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 627/1275] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:24:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 07:24:25.566042 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea424915-8baf-4505-a5c8-2320f9bb50d8 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:24:25.670748 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea424915-8baf-4505-a5c8-2320f9bb50d8 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:24:25.688703 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:24:25.705749 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2658b4-507b-4a65-b8a2-01af3fc0db71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:24:25.706379 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2658b4-507b-4a65-b8a2-01af3fc0db71 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:25.706661 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2658b4-507b-4a65-b8a2-01af3fc0db71 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:25.706935 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2658b4-507b-4a65-b8a2-01af3fc0db71 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:25.717624 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2658b4-507b-4a65-b8a2-01af3fc0db71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 07:24:25.724339 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-6a2658b4-507b-4a65-b8a2-01af3fc0db71 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 07:24:25.724339 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:24:25.724339 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:24:25.724339 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:24:25.724339 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:24:25.724339 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:24:25.724339 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:24:25.724339 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:24:25.724339 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:24:25.724339 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:24:25.724339 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:24:25.724339 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:24:25.724339 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 07:24:25.724339 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 07:24:25.724339 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 07:24:25.724339 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 07:24:25.725357 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 07:24:25.725357 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 07:24:25.725357 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 07:24:25.725357 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 07:24:25.725357 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:24:25.725357 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:24:25.725357 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:24:25.725357 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:24:25.725357 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:24:25.725357 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:24:25.725357 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:24:25.725357 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:24:25.725357 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:24:25.725357 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:24:25.725357 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:24:25.725357 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:24:25.726327 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:24:25.726327 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:24:25.726327 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:24:25.726327 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:24:25.726327 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 07:24:25.726327 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 07:24:25.726327 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 07:24:25.726327 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 07:24:25.726327 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 07:24:25.726327 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 07:24:25.726327 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 07:24:25.726327 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:24:25.726327 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 649/1276] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 15 07:24:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:24:25.729647 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f779d109-82d5-4fc1-a8f4-13bf6c0c281e None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:24:25.812207 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:24:25.830286 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f779d109-82d5-4fc1-a8f4-13bf6c0c281e None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:24:25.865344 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74046ba-ab71-4c78-9ff5-e4a217f91215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:24:25.865919 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74046ba-ab71-4c78-9ff5-e4a217f91215 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:25.866289 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74046ba-ab71-4c78-9ff5-e4a217f91215 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:25.866562 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74046ba-ab71-4c78-9ff5-e4a217f91215 None admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:25.867244 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74046ba-ab71-4c78-9ff5-e4a217f91215 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:24:25.867544 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74046ba-ab71-4c78-9ff5-e4a217f91215 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:24:25.868892 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74046ba-ab71-4c78-9ff5-e4a217f91215 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:24:25.882822 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 628/1277] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 15 07:24:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:24:25.884062 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2658b4-507b-4a65-b8a2-01af3fc0db71 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:24:25.989691 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2658b4-507b-4a65-b8a2-01af3fc0db71 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:24:26.023494 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595eac64-8937-45be-816f-20fced75f016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:24:26.024235 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595eac64-8937-45be-816f-20fced75f016 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:26.024575 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595eac64-8937-45be-816f-20fced75f016 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:26.024910 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595eac64-8937-45be-816f-20fced75f016 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:26.035526 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595eac64-8937-45be-816f-20fced75f016 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:24:26.036953 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595eac64-8937-45be-816f-20fced75f016 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:24:26.051471 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-595eac64-8937-45be-816f-20fced75f016 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:24:26.051471 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:24:26.051471 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:24:26.051471 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:24:26.051471 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:24:26.051471 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:24:26.051471 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:24:26.051471 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:24:26.051471 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:24:26.051471 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:24:26.051471 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:24:26.051471 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:24:26.051471 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 07:24:26.051471 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 07:24:26.051471 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 07:24:26.051471 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 07:24:26.052668 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:24:26.052668 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:24:26.052668 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:24:26.052668 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:24:26.052668 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:24:26.052668 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:24:26.052668 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:24:26.052668 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:24:26.052668 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:24:26.052668 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:24:26.052668 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:24:26.052668 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:24:26.052668 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:24:26.052668 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:24:26.052668 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:24:26.052668 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:24:26.052668 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 07:24:26.053595 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 07:24:26.053595 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:24:26.053595 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:24:26.053595 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 650/1278] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:24:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:24:26.058547 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74046ba-ab71-4c78-9ff5-e4a217f91215 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:24:26.160597 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74046ba-ab71-4c78-9ff5-e4a217f91215 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:24:26.197074 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5c9946-f3a3-43bc-834e-97d2b54569d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:24:26.197693 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5c9946-f3a3-43bc-834e-97d2b54569d1 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:26.197974 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5c9946-f3a3-43bc-834e-97d2b54569d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:26.198248 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5c9946-f3a3-43bc-834e-97d2b54569d1 None admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:26.198884 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5c9946-f3a3-43bc-834e-97d2b54569d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:24:26.199185 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5c9946-f3a3-43bc-834e-97d2b54569d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:24:26.200927 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5c9946-f3a3-43bc-834e-97d2b54569d1 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:24:26.212556 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 629/1279] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 15 07:24:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:24:26.218442 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595eac64-8937-45be-816f-20fced75f016 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:24:26.253153 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:24:26.333835 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595eac64-8937-45be-816f-20fced75f016 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:24:26.371495 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a276a88-0a3d-4f36-a83a-462f63ddd990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:24:26.372160 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a276a88-0a3d-4f36-a83a-462f63ddd990 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:26.372446 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a276a88-0a3d-4f36-a83a-462f63ddd990 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:26.372722 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a276a88-0a3d-4f36-a83a-462f63ddd990 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:26.381555 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a276a88-0a3d-4f36-a83a-462f63ddd990 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:24:26.383046 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a276a88-0a3d-4f36-a83a-462f63ddd990 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:24:26.395929 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-5a276a88-0a3d-4f36-a83a-462f63ddd990 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 15 07:24:26.395929 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:24:26.395929 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:24:26.395929 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:24:26.395929 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:24:26.395929 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:24:26.395929 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:24:26.395929 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:24:26.395929 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:24:26.395929 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:24:26.395929 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:24:26.395929 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:24:26.395929 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 07:24:26.395929 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 07:24:26.395929 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 07:24:26.395929 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 07:24:26.397141 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:24:26.397141 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:24:26.397141 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 07:24:26.397141 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 07:24:26.397141 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 07:24:26.397141 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 07:24:26.397141 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:24:26.397141 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:24:26.397141 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:24:26.397141 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:24:26.397141 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:24:26.397141 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:24:26.397141 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:24:26.397141 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:24:26.397141 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:24:26.397141 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:24:26.397141 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:24:26.397141 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:24:26.398272 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:24:26.398272 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:24:26.398272 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 07:24:26.398272 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 07:24:26.398272 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 07:24:26.398272 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 07:24:26.398272 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 07:24:26.398272 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 07:24:26.398272 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 15 07:24:26.398272 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:24:26.398272 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 651/1280] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:24:26 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:24:26.403069 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5c9946-f3a3-43bc-834e-97d2b54569d1 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:24:26.407613 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:24:26.515779 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5c9946-f3a3-43bc-834e-97d2b54569d1 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:24:26.554016 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b0620c-c5ea-4713-be75-0e3afcfa3975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:24:26.554648 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b0620c-c5ea-4713-be75-0e3afcfa3975 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:26.554929 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b0620c-c5ea-4713-be75-0e3afcfa3975 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:26.555201 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b0620c-c5ea-4713-be75-0e3afcfa3975 None admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:26.555936 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b0620c-c5ea-4713-be75-0e3afcfa3975 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:24:26.556245 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b0620c-c5ea-4713-be75-0e3afcfa3975 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:24:26.557818 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b0620c-c5ea-4713-be75-0e3afcfa3975 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:24:26.579581 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 630/1281] 10.222.0.42 () {56 vars in 1143 bytes} [Thu Sep 15 07:24:26 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:24:26.585888 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a276a88-0a3d-4f36-a83a-462f63ddd990 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:24:26.679960 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:24:26.689678 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a276a88-0a3d-4f36-a83a-462f63ddd990 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:24:26.724902 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c7b292-11b4-4957-8194-0cd1f44d4a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:24:26.725506 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c7b292-11b4-4957-8194-0cd1f44d4a6b None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:26.725810 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c7b292-11b4-4957-8194-0cd1f44d4a6b None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:26.726080 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c7b292-11b4-4957-8194-0cd1f44d4a6b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:26.793222 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c7b292-11b4-4957-8194-0cd1f44d4a6b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:24:26.794716 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c7b292-11b4-4957-8194-0cd1f44d4a6b None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:24:26.884092 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:24:26.896875 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 652/1282] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 15 07:24:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 07:24:26.902981 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b0620c-c5ea-4713-be75-0e3afcfa3975 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:24:26.964743 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:24:27.002955 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b0620c-c5ea-4713-be75-0e3afcfa3975 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:24:27.038376 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3453694-721b-4cdf-b855-f7976d1b7f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:24:27.039389 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3453694-721b-4cdf-b855-f7976d1b7f21 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:27.039851 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3453694-721b-4cdf-b855-f7976d1b7f21 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:27.040302 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3453694-721b-4cdf-b855-f7976d1b7f21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:27.051400 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3453694-721b-4cdf-b855-f7976d1b7f21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:24:27.052798 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3453694-721b-4cdf-b855-f7976d1b7f21 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:24:27.067931 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-a3453694-721b-4cdf-b855-f7976d1b7f21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:24:27.067931 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:24:27.067931 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:24:27.067931 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:24:27.067931 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:24:27.067931 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:24:27.067931 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:24:27.067931 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:24:27.067931 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:24:27.067931 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:24:27.067931 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:24:27.067931 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:24:27.067931 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 07:24:27.067931 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 07:24:27.067931 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 07:24:27.067931 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 07:24:27.069026 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:24:27.069026 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:24:27.069026 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:24:27.069026 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:24:27.069026 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:24:27.069026 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:24:27.069026 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:24:27.069026 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:24:27.069026 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:24:27.069026 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:24:27.069026 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:24:27.069026 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:24:27.069026 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:24:27.069026 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:24:27.069026 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:24:27.069026 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:24:27.069026 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 07:24:27.070132 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 07:24:27.070132 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 07:24:27.070132 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:24:27.070132 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 631/1283] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:24:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:24:27.075107 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c7b292-11b4-4957-8194-0cd1f44d4a6b None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:24:27.172381 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c7b292-11b4-4957-8194-0cd1f44d4a6b None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:24:27.206271 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0239b9-b917-4f7d-87c3-6597513a3dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:24:27.206871 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0239b9-b917-4f7d-87c3-6597513a3dbe None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:27.207149 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0239b9-b917-4f7d-87c3-6597513a3dbe None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:27.207418 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0239b9-b917-4f7d-87c3-6597513a3dbe None admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:27.208044 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0239b9-b917-4f7d-87c3-6597513a3dbe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:24:27.208345 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0239b9-b917-4f7d-87c3-6597513a3dbe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:24:27.209883 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0239b9-b917-4f7d-87c3-6597513a3dbe None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:24:27.221663 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 653/1284] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 15 07:24:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:24:27.227530 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3453694-721b-4cdf-b855-f7976d1b7f21 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:24:27.327462 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3453694-721b-4cdf-b855-f7976d1b7f21 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:24:27.340459 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:24:27.360242 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dc7923-77ac-498a-8b8e-ee21df6686c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:24:27.360865 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dc7923-77ac-498a-8b8e-ee21df6686c0 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:27.361152 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dc7923-77ac-498a-8b8e-ee21df6686c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:27.361431 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dc7923-77ac-498a-8b8e-ee21df6686c0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:27.368157 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dc7923-77ac-498a-8b8e-ee21df6686c0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:24:27.369909 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dc7923-77ac-498a-8b8e-ee21df6686c0 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:24:27.382194 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application [None req-b6dc7923-77ac-498a-8b8e-ee21df6686c0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 07:24:27.382194 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:24:27.382194 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:24:27.382194 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:24:27.382194 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:24:27.382194 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:24:27.382194 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:24:27.382194 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:24:27.382194 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:24:27.382194 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:24:27.382194 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:24:27.382194 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:24:27.382194 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 07:24:27.382194 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 07:24:27.382194 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 07:24:27.383323 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 07:24:27.383323 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:24:27.383323 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:24:27.383323 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:24:27.383323 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:24:27.383323 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:24:27.383323 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:24:27.383323 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:24:27.383323 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application with Lock( Sep 15 07:24:27.383323 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:24:27.383323 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:24:27.383323 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:24:27.383323 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:24:27.383323 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:24:27.383323 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:24:27.383323 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:24:27.383323 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:24:27.384242 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 07:24:27.384242 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 07:24:27.384242 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 07:24:27.384242 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 07:24:27.384242 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 07:24:27.384242 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 07:24:27.384242 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 07:24:27.384242 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: ERROR keystone.server.flask.application Sep 15 07:24:27.384242 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 632/1285] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 15 07:24:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:24:27.389927 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0239b9-b917-4f7d-87c3-6597513a3dbe None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:24:27.439732 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:24:27.489323 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0239b9-b917-4f7d-87c3-6597513a3dbe None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:24:27.521616 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc500b8-a145-4194-9a31-89da81234a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:24:27.522219 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc500b8-a145-4194-9a31-89da81234a68 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:27.522500 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc500b8-a145-4194-9a31-89da81234a68 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:27.522774 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc500b8-a145-4194-9a31-89da81234a68 None admin] PATH_INFO: `/v3/projects` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:27.523449 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc500b8-a145-4194-9a31-89da81234a68 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:24:27.523753 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc500b8-a145-4194-9a31-89da81234a68 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:24:27.525160 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc500b8-a145-4194-9a31-89da81234a68 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:24:27.537312 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 654/1286] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 15 07:24:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:24:27.543521 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dc7923-77ac-498a-8b8e-ee21df6686c0 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:24:27.649680 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dc7923-77ac-498a-8b8e-ee21df6686c0 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:24:27.682540 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46628d71-43a7-4564-940d-dda2120e5ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:24:27.683131 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46628d71-43a7-4564-940d-dda2120e5ab2 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:27.683410 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46628d71-43a7-4564-940d-dda2120e5ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:27.683678 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46628d71-43a7-4564-940d-dda2120e5ab2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:27.684389 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46628d71-43a7-4564-940d-dda2120e5ab2 None admin] RBAC: Adding query filter params (role.id=6bbb643e0d8d4bc0a9b27c26edf59216, scope.project.id=4e9987720930473da840579654311bc2, user.id=dd87846b9a134248b28e0521b0a880c9) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:24:27.684691 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46628d71-43a7-4564-940d-dda2120e5ab2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:24:27.686311 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46628d71-43a7-4564-940d-dda2120e5ab2 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:24:27.691991 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 633/1287] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 15 07:24:27 2022] GET /identity/v3/role_assignments?user.id=dd87846b9a134248b28e0521b0a880c9&scope.project.id=4e9987720930473da840579654311bc2&role.id=6bbb643e0d8d4bc0a9b27c26edf59216 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:24:29.858280 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74707f7-6654-40ae-a5af-b2e9dc436416 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:29.858836 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74707f7-6654-40ae-a5af-b2e9dc436416 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:29.858836 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74707f7-6654-40ae-a5af-b2e9dc436416 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:29.860136 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 655/1288] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:24:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:24:29.865282 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050e44ca-c5d4-4b03-a893-5b866e699e4c None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:29.865692 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050e44ca-c5d4-4b03-a893-5b866e699e4c None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:29.865867 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050e44ca-c5d4-4b03-a893-5b866e699e4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:29.884364 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-050e44ca-c5d4-4b03-a893-5b866e699e4c None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:24:29.885533 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-050e44ca-c5d4-4b03-a893-5b866e699e4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:24:29.915353 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:24:29.953042 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 634/1289] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:24:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 07:24:30.219849 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74707f7-6654-40ae-a5af-b2e9dc436416 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:24:30.333675 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74707f7-6654-40ae-a5af-b2e9dc436416 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:24:30.344712 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:24:30.367630 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec872860-0dc3-40fd-a9b1-0bc1abb04fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:24:30.368248 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec872860-0dc3-40fd-a9b1-0bc1abb04fc8 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:30.368549 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec872860-0dc3-40fd-a9b1-0bc1abb04fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:30.368821 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec872860-0dc3-40fd-a9b1-0bc1abb04fc8 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:30.369387 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec872860-0dc3-40fd-a9b1-0bc1abb04fc8 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:24:30.370781 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec872860-0dc3-40fd-a9b1-0bc1abb04fc8 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:24:30.379954 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-ec872860-0dc3-40fd-a9b1-0bc1abb04fc8 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 15 07:24:30.379954 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:24:30.379954 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 07:24:30.379954 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 07:24:30.379954 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 07:24:30.379954 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 07:24:30.379954 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 07:24:30.379954 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 07:24:30.379954 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 15 07:24:30.379954 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:24:30.379954 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 07:24:30.379954 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:24:30.379954 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:24:30.379954 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:24:30.379954 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:24:30.379954 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:24:30.379954 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:24:30.381301 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:24:30.381301 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:24:30.381301 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:24:30.381301 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:24:30.381301 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:24:30.381301 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:24:30.381301 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 07:24:30.381301 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 07:24:30.381301 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 07:24:30.381301 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 07:24:30.381301 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:24:30.381301 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:24:30.381301 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:24:30.381301 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:24:30.381301 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:24:30.381301 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:24:30.382485 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:24:30.382485 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:24:30.382485 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:24:30.382485 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:24:30.382485 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:24:30.382485 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:24:30.382485 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:24:30.382485 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:24:30.382485 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:24:30.382485 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:24:30.382485 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 07:24:30.382485 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 07:24:30.382485 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 15 07:24:30.382485 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:24:30.382485 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 656/1290] 10.222.0.42 () {56 vars in 1117 bytes} [Thu Sep 15 07:24:30 2022] GET /identity/v3/services/placement => generated 90 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:24:30.387032 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050e44ca-c5d4-4b03-a893-5b866e699e4c None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:24:30.483318 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050e44ca-c5d4-4b03-a893-5b866e699e4c None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:24:30.515873 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7668c00-050d-4540-94d8-f2fd204d435c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:24:30.516470 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7668c00-050d-4540-94d8-f2fd204d435c None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:30.516745 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7668c00-050d-4540-94d8-f2fd204d435c None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:30.517014 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7668c00-050d-4540-94d8-f2fd204d435c None admin] PATH_INFO: `/v3/services` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:30.517651 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7668c00-050d-4540-94d8-f2fd204d435c None admin] RBAC: Adding query filter params (name=placement) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:24:30.517953 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7668c00-050d-4540-94d8-f2fd204d435c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:24:30.519313 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7668c00-050d-4540-94d8-f2fd204d435c None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:24:30.523989 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 635/1291] 10.222.0.42 () {56 vars in 1116 bytes} [Thu Sep 15 07:24:30 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:24:30.530248 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec872860-0dc3-40fd-a9b1-0bc1abb04fc8 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:24:30.631479 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec872860-0dc3-40fd-a9b1-0bc1abb04fc8 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:24:30.665774 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c58ab1d-cd0c-49e8-813a-bf319daa98d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:24:30.666377 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c58ab1d-cd0c-49e8-813a-bf319daa98d0 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:30.666657 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c58ab1d-cd0c-49e8-813a-bf319daa98d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:30.666933 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c58ab1d-cd0c-49e8-813a-bf319daa98d0 None admin] PATH_INFO: `/v3/services` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:30.667547 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c58ab1d-cd0c-49e8-813a-bf319daa98d0 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:24:30.667849 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c58ab1d-cd0c-49e8-813a-bf319daa98d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:24:30.669217 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c58ab1d-cd0c-49e8-813a-bf319daa98d0 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:24:30.673998 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 657/1292] 10.222.0.42 () {56 vars in 1116 bytes} [Thu Sep 15 07:24:30 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:24:32.831783 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dd1b90-75c9-4ef0-8833-ffaecb1880e7 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:32.832314 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dd1b90-75c9-4ef0-8833-ffaecb1880e7 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:32.832314 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dd1b90-75c9-4ef0-8833-ffaecb1880e7 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:32.833697 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 636/1293] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:24:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:24:32.839161 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b8276f-b614-423b-94c4-f6b9d091abcb None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:32.839587 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b8276f-b614-423b-94c4-f6b9d091abcb None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:32.839711 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b8276f-b614-423b-94c4-f6b9d091abcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:32.858269 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-b6b8276f-b614-423b-94c4-f6b9d091abcb None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:24:32.862744 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-b6b8276f-b614-423b-94c4-f6b9d091abcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:24:32.893028 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:24:32.934880 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 658/1294] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:24:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 07:24:33.206834 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dd1b90-75c9-4ef0-8833-ffaecb1880e7 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:24:33.238663 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:24:33.307515 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dd1b90-75c9-4ef0-8833-ffaecb1880e7 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:24:33.340170 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369b6e4e-423d-4f1f-a5e3-1afa9fe35ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:24:33.340766 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369b6e4e-423d-4f1f-a5e3-1afa9fe35ed3 None admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:33.341045 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369b6e4e-423d-4f1f-a5e3-1afa9fe35ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:33.341317 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369b6e4e-423d-4f1f-a5e3-1afa9fe35ed3 None admin] PATH_INFO: `/v3/services` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:33.342000 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369b6e4e-423d-4f1f-a5e3-1afa9fe35ed3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:24:33.344031 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369b6e4e-423d-4f1f-a5e3-1afa9fe35ed3 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:24:33.419029 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 637/1295] 10.222.0.42 () {60 vars in 1141 bytes} [Thu Sep 15 07:24:33 2022] POST /identity/v3/services => generated 246 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:24:35.599290 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deee8f30-c211-4819-a7e8-6756f7ceddd8 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:35.599829 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deee8f30-c211-4819-a7e8-6756f7ceddd8 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:35.599829 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deee8f30-c211-4819-a7e8-6756f7ceddd8 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:35.601157 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 659/1296] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:24:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:24:35.606485 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27cb6ac-b965-4075-ad3f-936e1ad8912f None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:35.606914 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27cb6ac-b965-4075-ad3f-936e1ad8912f None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:35.607043 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27cb6ac-b965-4075-ad3f-936e1ad8912f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:35.625870 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-c27cb6ac-b965-4075-ad3f-936e1ad8912f None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:24:35.627055 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-c27cb6ac-b965-4075-ad3f-936e1ad8912f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:24:35.699072 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 638/1297] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:24:35 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 07:24:35.973299 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deee8f30-c211-4819-a7e8-6756f7ceddd8 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:24:36.073364 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deee8f30-c211-4819-a7e8-6756f7ceddd8 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:24:36.106829 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6718396e-527e-4b6a-8aab-a9fb29a2744e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:24:36.107489 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6718396e-527e-4b6a-8aab-a9fb29a2744e None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:36.107744 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6718396e-527e-4b6a-8aab-a9fb29a2744e None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:36.108589 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6718396e-527e-4b6a-8aab-a9fb29a2744e None admin] PATH_INFO: `/v3/services/placement` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:36.109172 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6718396e-527e-4b6a-8aab-a9fb29a2744e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:24:36.110570 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6718396e-527e-4b6a-8aab-a9fb29a2744e None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:24:36.119281 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-6718396e-527e-4b6a-8aab-a9fb29a2744e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 15 07:24:36.119281 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:24:36.119281 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 07:24:36.119281 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 07:24:36.119281 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 07:24:36.119281 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 07:24:36.119281 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 07:24:36.119281 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 07:24:36.119281 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 15 07:24:36.119281 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:24:36.119281 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 07:24:36.119281 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:24:36.119281 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:24:36.119281 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:24:36.119281 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:24:36.119281 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:24:36.119281 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:24:36.120445 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:24:36.120445 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:24:36.120445 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:24:36.120445 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:24:36.120445 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:24:36.120445 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:24:36.120445 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 07:24:36.120445 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 07:24:36.120445 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 07:24:36.120445 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 07:24:36.120445 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:24:36.120445 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:24:36.120445 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:24:36.120445 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:24:36.120445 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:24:36.120445 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:24:36.121486 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:24:36.121486 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:24:36.121486 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:24:36.121486 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:24:36.121486 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:24:36.121486 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:24:36.121486 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:24:36.121486 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:24:36.121486 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:24:36.121486 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:24:36.121486 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 07:24:36.121486 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 07:24:36.121486 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 15 07:24:36.121486 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:24:36.121486 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 660/1298] 10.222.0.42 () {56 vars in 1117 bytes} [Thu Sep 15 07:24:35 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:24:36.127361 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27cb6ac-b965-4075-ad3f-936e1ad8912f None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:24:36.145535 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:24:36.225214 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27cb6ac-b965-4075-ad3f-936e1ad8912f None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:24:36.257852 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f6d7fb-ed08-467e-a1fa-124eb23ded13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:24:36.258452 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f6d7fb-ed08-467e-a1fa-124eb23ded13 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:36.258730 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f6d7fb-ed08-467e-a1fa-124eb23ded13 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:36.258996 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f6d7fb-ed08-467e-a1fa-124eb23ded13 None admin] PATH_INFO: `/v3/services` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:36.259614 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f6d7fb-ed08-467e-a1fa-124eb23ded13 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:24:36.259943 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f6d7fb-ed08-467e-a1fa-124eb23ded13 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:24:36.261455 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f6d7fb-ed08-467e-a1fa-124eb23ded13 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:24:36.267455 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 639/1299] 10.222.0.42 () {56 vars in 1116 bytes} [Thu Sep 15 07:24:36 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:24:36.273641 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6718396e-527e-4b6a-8aab-a9fb29a2744e None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:24:36.305461 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:24:36.371902 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6718396e-527e-4b6a-8aab-a9fb29a2744e None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:24:36.406784 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5611f240-c8fd-45b0-9f10-d3e3dd5f3f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:24:36.407406 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5611f240-c8fd-45b0-9f10-d3e3dd5f3f47 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:36.407699 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5611f240-c8fd-45b0-9f10-d3e3dd5f3f47 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:36.407974 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5611f240-c8fd-45b0-9f10-d3e3dd5f3f47 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:36.408669 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5611f240-c8fd-45b0-9f10-d3e3dd5f3f47 None admin] RBAC: Adding query filter params (interface=public, service_id=e9b323c5b6474a129d640b3766279bbb, region_id=RegionOne) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:24:36.409043 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5611f240-c8fd-45b0-9f10-d3e3dd5f3f47 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:24:36.410496 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5611f240-c8fd-45b0-9f10-d3e3dd5f3f47 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:24:36.415412 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 661/1300] 10.222.0.42 () {56 vars in 1251 bytes} [Thu Sep 15 07:24:36 2022] GET /identity/v3/endpoints?service_id=e9b323c5b6474a129d640b3766279bbb&interface=public®ion_id=RegionOne => generated 194 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:24:36.421779 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f6d7fb-ed08-467e-a1fa-124eb23ded13 None admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:24:36.502367 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:24:36.519501 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f6d7fb-ed08-467e-a1fa-124eb23ded13 None admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:24:36.553132 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18630018-fe58-4276-94f6-e755c9470888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:24:36.553796 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18630018-fe58-4276-94f6-e755c9470888 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:36.554294 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18630018-fe58-4276-94f6-e755c9470888 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:36.554999 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18630018-fe58-4276-94f6-e755c9470888 None admin] PATH_INFO: `/v3/services` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:36.555627 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18630018-fe58-4276-94f6-e755c9470888 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:24:36.556989 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18630018-fe58-4276-94f6-e755c9470888 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:24:36.563826 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 640/1301] 10.222.0.42 () {56 vars in 1087 bytes} [Thu Sep 15 07:24:36 2022] GET /identity/v3/services => generated 2191 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:24:38.715711 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b293ddbf-7d97-449f-9b34-15bfe597cfc7 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:38.716238 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b293ddbf-7d97-449f-9b34-15bfe597cfc7 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:38.716332 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b293ddbf-7d97-449f-9b34-15bfe597cfc7 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:38.718389 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 662/1302] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:24:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:24:38.726198 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686e014f-c386-4290-9313-cfe4184db90f None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:38.726633 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686e014f-c386-4290-9313-cfe4184db90f None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:38.726915 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686e014f-c386-4290-9313-cfe4184db90f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:38.745787 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-686e014f-c386-4290-9313-cfe4184db90f None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:24:38.747134 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-686e014f-c386-4290-9313-cfe4184db90f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:24:38.815119 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 641/1303] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:24:38 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 07:24:39.096150 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b293ddbf-7d97-449f-9b34-15bfe597cfc7 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:24:39.193853 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:24:39.200378 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b293ddbf-7d97-449f-9b34-15bfe597cfc7 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:24:39.235121 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9268791b-3b42-4cb4-8684-1d7d51520d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:24:39.235758 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9268791b-3b42-4cb4-8684-1d7d51520d3d None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:39.236036 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9268791b-3b42-4cb4-8684-1d7d51520d3d None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:39.236308 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9268791b-3b42-4cb4-8684-1d7d51520d3d None admin] PATH_INFO: `/v3/services/placement` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:39.236882 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9268791b-3b42-4cb4-8684-1d7d51520d3d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:24:39.238264 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9268791b-3b42-4cb4-8684-1d7d51520d3d None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:24:39.247069 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-9268791b-3b42-4cb4-8684-1d7d51520d3d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 15 07:24:39.247069 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:24:39.247069 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 07:24:39.247069 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 07:24:39.247069 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 07:24:39.247069 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 07:24:39.247069 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 07:24:39.247069 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 07:24:39.247069 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 15 07:24:39.247069 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:24:39.247069 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 07:24:39.247069 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:24:39.247069 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 07:24:39.247069 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 07:24:39.247069 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 07:24:39.247069 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 07:24:39.247069 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 07:24:39.248247 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 07:24:39.248247 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 07:24:39.248247 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 07:24:39.248247 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 07:24:39.248247 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 07:24:39.248247 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 07:24:39.248247 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 07:24:39.248247 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 07:24:39.248247 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 07:24:39.248247 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 07:24:39.248247 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 07:24:39.248247 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 07:24:39.248247 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 07:24:39.248247 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 07:24:39.248247 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 07:24:39.248247 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 07:24:39.249937 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 07:24:39.249937 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Sep 15 07:24:39.249937 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 07:24:39.249937 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Sep 15 07:24:39.249937 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 07:24:39.249937 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 07:24:39.249937 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 07:24:39.249937 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Sep 15 07:24:39.249937 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 07:24:39.249937 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Sep 15 07:24:39.249937 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 07:24:39.249937 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 07:24:39.249937 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 15 07:24:39.249937 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Sep 15 07:24:39.249937 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 663/1304] 10.222.0.42 () {56 vars in 1117 bytes} [Thu Sep 15 07:24:39 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 07:24:39.255478 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686e014f-c386-4290-9313-cfe4184db90f None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:24:39.331168 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:24:39.352825 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686e014f-c386-4290-9313-cfe4184db90f None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:24:39.385513 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3e6adc-7dba-4650-aa71-16cf82a0f5d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:24:39.386147 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3e6adc-7dba-4650-aa71-16cf82a0f5d3 None admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:39.386460 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3e6adc-7dba-4650-aa71-16cf82a0f5d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:39.386740 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3e6adc-7dba-4650-aa71-16cf82a0f5d3 None admin] PATH_INFO: `/v3/services` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:39.387363 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3e6adc-7dba-4650-aa71-16cf82a0f5d3 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:24:39.387667 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3e6adc-7dba-4650-aa71-16cf82a0f5d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:24:39.389016 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3e6adc-7dba-4650-aa71-16cf82a0f5d3 None admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:24:39.394820 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 642/1305] 10.222.0.42 () {56 vars in 1116 bytes} [Thu Sep 15 07:24:39 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:24:39.400628 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9268791b-3b42-4cb4-8684-1d7d51520d3d None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:24:39.500643 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9268791b-3b42-4cb4-8684-1d7d51520d3d None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:24:39.533896 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2678be21-0e83-43de-a8ef-f118b1f4f537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:24:39.534496 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2678be21-0e83-43de-a8ef-f118b1f4f537 None admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:24:39.534776 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2678be21-0e83-43de-a8ef-f118b1f4f537 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:24:39.535048 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2678be21-0e83-43de-a8ef-f118b1f4f537 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:24:39.535709 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2678be21-0e83-43de-a8ef-f118b1f4f537 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:24:39.537069 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2678be21-0e83-43de-a8ef-f118b1f4f537 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:24:39.672295 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 664/1306] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:24:39 2022] POST /identity/v3/endpoints => generated 331 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:24:49.431053 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=30 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:24:49.666759 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:30:25.600534 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f59e24b-e586-44d4-84a3-134ae7c10773 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:30:25.600972 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f59e24b-e586-44d4-84a3-134ae7c10773 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:30:25.600972 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f59e24b-e586-44d4-84a3-134ae7c10773 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:30:25.602391 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 643/1307] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:30:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:30:25.610851 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1c2353-673e-4dff-bb53-31b8ed3d80e5 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:30:25.611329 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1c2353-673e-4dff-bb53-31b8ed3d80e5 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:30:25.611408 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1c2353-673e-4dff-bb53-31b8ed3d80e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:30:25.720401 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-ed1c2353-673e-4dff-bb53-31b8ed3d80e5 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:30:25.722245 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-ed1c2353-673e-4dff-bb53-31b8ed3d80e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:30:25.838513 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 665/1308] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:30:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:30:26.136211 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e82b370-bdea-4e4c-9503-6222fadfbfa2 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:30:26.136521 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e82b370-bdea-4e4c-9503-6222fadfbfa2 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:30:26.136797 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e82b370-bdea-4e4c-9503-6222fadfbfa2 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:30:26.138170 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 644/1309] 10.222.0.42 () {54 vars in 965 bytes} [Thu Sep 15 07:30:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:30:26.147662 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a649a9c-1c0e-45a4-b986-e0fb1ac289f4 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:30:26.148217 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a649a9c-1c0e-45a4-b986-e0fb1ac289f4 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:30:26.148643 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a649a9c-1c0e-45a4-b986-e0fb1ac289f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:30:26.228558 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-0a649a9c-1c0e-45a4-b986-e0fb1ac289f4 None None] MFA Rules not processed for user `5315996432f44bffa3460b2744c10f0a`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:30:26.230707 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-0a649a9c-1c0e-45a4-b986-e0fb1ac289f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:30:26.256512 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:30:26.315739 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 666/1310] 10.222.0.42 () {58 vars in 1062 bytes} [Thu Sep 15 07:30:26 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 07:30:26.325080 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e82b370-bdea-4e4c-9503-6222fadfbfa2 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:30:26.326926 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-3e82b370-bdea-4e4c-9503-6222fadfbfa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:30:26.491184 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e82b370-bdea-4e4c-9503-6222fadfbfa2 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:30:26.591790 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4c1166-197b-430d-b215-550d1bc4de64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:30:26.592437 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c1166-197b-430d-b215-550d1bc4de64 service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:30:26.592716 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c1166-197b-430d-b215-550d1bc4de64 service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:30:26.592987 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c1166-197b-430d-b215-550d1bc4de64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:30:26.594605 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-0c4c1166-197b-430d-b215-550d1bc4de64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:30:26.661282 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:30:26.730793 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4c1166-197b-430d-b215-550d1bc4de64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:30:26.732244 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4c1166-197b-430d-b215-550d1bc4de64 service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:30:26.767869 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 645/1311] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:30:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:30:29.317218 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd6cedd-fa18-4119-bbfe-476e2be4c779 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:30:29.317715 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd6cedd-fa18-4119-bbfe-476e2be4c779 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:30:29.317827 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd6cedd-fa18-4119-bbfe-476e2be4c779 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:30:29.319150 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 667/1312] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:30:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:30:29.324398 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0644aac1-4b2d-4633-a5e8-90694e2e3161 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:30:29.324800 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0644aac1-4b2d-4633-a5e8-90694e2e3161 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:30:29.324993 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0644aac1-4b2d-4633-a5e8-90694e2e3161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:30:29.420641 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-0644aac1-4b2d-4633-a5e8-90694e2e3161 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:30:29.421888 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-0644aac1-4b2d-4633-a5e8-90694e2e3161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:30:29.519814 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 646/1313] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:30:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:30:29.802182 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd6cedd-fa18-4119-bbfe-476e2be4c779 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:30:29.803715 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-6cd6cedd-fa18-4119-bbfe-476e2be4c779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:30:29.909434 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd6cedd-fa18-4119-bbfe-476e2be4c779 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:30:29.983973 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:30:29.986234 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abe0c22-c628-4400-8a5c-7a6cee82d594 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:30:29.987382 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abe0c22-c628-4400-8a5c-7a6cee82d594 service neutron] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:30:29.987661 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abe0c22-c628-4400-8a5c-7a6cee82d594 service neutron] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:30:29.988025 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abe0c22-c628-4400-8a5c-7a6cee82d594 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:30:29.989515 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-3abe0c22-c628-4400-8a5c-7a6cee82d594 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:30:30.099730 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abe0c22-c628-4400-8a5c-7a6cee82d594 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:30:30.101255 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abe0c22-c628-4400-8a5c-7a6cee82d594 service neutron] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:30:30.120355 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 668/1314] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:30:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:30:32.839068 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852c3348-e360-470a-bf65-0cfbaaf64b9e None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:30:32.839699 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852c3348-e360-470a-bf65-0cfbaaf64b9e None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:30:32.839699 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852c3348-e360-470a-bf65-0cfbaaf64b9e None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:30:32.840972 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 647/1315] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:30:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:30:32.846651 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3b0971-eeb9-4077-9eba-1617a63533b6 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:30:32.847105 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3b0971-eeb9-4077-9eba-1617a63533b6 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:30:32.847215 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3b0971-eeb9-4077-9eba-1617a63533b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:30:32.916298 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-bf3b0971-eeb9-4077-9eba-1617a63533b6 None None] MFA Rules not processed for user `001806a30226487ab9e17f8d988f1ca6`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:30:32.917519 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-bf3b0971-eeb9-4077-9eba-1617a63533b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:30:32.973010 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:30:33.019166 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 669/1316] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:30:32 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:30:33.324823 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c78afd1-656e-49bd-9c5a-361c7732353d None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:30:33.325161 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c78afd1-656e-49bd-9c5a-361c7732353d None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:30:33.325437 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c78afd1-656e-49bd-9c5a-361c7732353d None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:30:33.326857 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 648/1317] 10.222.0.42 () {54 vars in 965 bytes} [Thu Sep 15 07:30:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 07:30:33.333206 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f873350b-0bc4-4128-9e35-b1eea0c91ce7 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:30:33.333569 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f873350b-0bc4-4128-9e35-b1eea0c91ce7 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:30:33.333947 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f873350b-0bc4-4128-9e35-b1eea0c91ce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:30:33.412990 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-f873350b-0bc4-4128-9e35-b1eea0c91ce7 None None] MFA Rules not processed for user `5315996432f44bffa3460b2744c10f0a`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:30:33.414222 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-f873350b-0bc4-4128-9e35-b1eea0c91ce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:30:33.493519 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 670/1318] 10.222.0.42 () {58 vars in 1062 bytes} [Thu Sep 15 07:30:33 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 07:30:33.503312 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c78afd1-656e-49bd-9c5a-361c7732353d None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:30:33.504826 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-5c78afd1-656e-49bd-9c5a-361c7732353d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:30:33.515267 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:30:33.616662 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c78afd1-656e-49bd-9c5a-361c7732353d None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:30:33.693389 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a568fa1-b0f8-417e-96e6-c8f5a3c1425d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:30:33.694054 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a568fa1-b0f8-417e-96e6-c8f5a3c1425d service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:30:33.694336 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a568fa1-b0f8-417e-96e6-c8f5a3c1425d service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:30:33.694611 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a568fa1-b0f8-417e-96e6-c8f5a3c1425d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:30:33.696210 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-9a568fa1-b0f8-417e-96e6-c8f5a3c1425d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:30:33.768855 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:30:33.818319 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a568fa1-b0f8-417e-96e6-c8f5a3c1425d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:30:33.819745 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a568fa1-b0f8-417e-96e6-c8f5a3c1425d service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:30:33.838693 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 649/1319] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:30:33 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:30:37.262093 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c393c6-9c6e-49b2-a008-3aebe095d0f2 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:30:37.262612 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c393c6-9c6e-49b2-a008-3aebe095d0f2 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:30:37.262709 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c393c6-9c6e-49b2-a008-3aebe095d0f2 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:30:37.263976 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 671/1320] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:30:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:30:37.270058 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbbafb5-233c-4401-9672-7a294417191a None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:30:37.270512 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbbafb5-233c-4401-9672-7a294417191a None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:30:37.270615 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbbafb5-233c-4401-9672-7a294417191a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:30:37.340009 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-4cbbafb5-233c-4401-9672-7a294417191a None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:30:37.341210 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-4cbbafb5-233c-4401-9672-7a294417191a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:30:37.440276 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 650/1321] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:30:37 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:30:37.729167 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c393c6-9c6e-49b2-a008-3aebe095d0f2 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:30:37.730985 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-12c393c6-9c6e-49b2-a008-3aebe095d0f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:30:37.784731 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:30:37.838182 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c393c6-9c6e-49b2-a008-3aebe095d0f2 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:30:37.920454 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c33d30c-3864-44b4-98ed-155506d5540b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:30:37.921061 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c33d30c-3864-44b4-98ed-155506d5540b service neutron] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:30:37.921339 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c33d30c-3864-44b4-98ed-155506d5540b service neutron] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:30:37.921630 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c33d30c-3864-44b4-98ed-155506d5540b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:30:37.923336 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-3c33d30c-3864-44b4-98ed-155506d5540b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:30:38.026095 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:30:38.040502 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c33d30c-3864-44b4-98ed-155506d5540b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:30:38.041990 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c33d30c-3864-44b4-98ed-155506d5540b service neutron] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:30:38.059683 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 672/1322] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:30:37 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:30:41.289141 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a711e61b-0485-4a8a-a447-0e99a9bbc440 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:30:41.289682 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a711e61b-0485-4a8a-a447-0e99a9bbc440 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:30:41.289859 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a711e61b-0485-4a8a-a447-0e99a9bbc440 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:30:41.291280 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 651/1323] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:30:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:30:41.296597 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaec52b-27d0-4951-94cd-70c4190af8d6 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:30:41.296980 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaec52b-27d0-4951-94cd-70c4190af8d6 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:30:41.297140 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaec52b-27d0-4951-94cd-70c4190af8d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:30:41.365098 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-2aaec52b-27d0-4951-94cd-70c4190af8d6 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:30:41.366331 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-2aaec52b-27d0-4951-94cd-70c4190af8d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:30:41.467268 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 673/1324] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:30:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:30:41.752368 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a711e61b-0485-4a8a-a447-0e99a9bbc440 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:30:41.754071 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-a711e61b-0485-4a8a-a447-0e99a9bbc440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:30:41.773634 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:30:41.858657 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a711e61b-0485-4a8a-a447-0e99a9bbc440 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:30:41.935607 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df0438d-1881-4020-885b-31449a860690 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:30:41.936218 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df0438d-1881-4020-885b-31449a860690 service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:30:41.936495 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df0438d-1881-4020-885b-31449a860690 service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:30:41.936765 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df0438d-1881-4020-885b-31449a860690 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:30:41.941043 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-8df0438d-1881-4020-885b-31449a860690 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:30:42.012345 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:30:42.051063 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df0438d-1881-4020-885b-31449a860690 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:30:42.052479 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df0438d-1881-4020-885b-31449a860690 service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:30:42.073952 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 652/1325] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:30:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:30:44.331499 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf46c5a0-4f57-4631-bad3-adbcc4559784 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:30:44.332236 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf46c5a0-4f57-4631-bad3-adbcc4559784 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:30:44.332236 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf46c5a0-4f57-4631-bad3-adbcc4559784 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:30:44.333514 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 674/1326] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:30:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:30:44.338837 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86c23f0-61d4-4a33-95e6-f94c7b56fc90 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:30:44.339254 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86c23f0-61d4-4a33-95e6-f94c7b56fc90 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:30:44.339394 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86c23f0-61d4-4a33-95e6-f94c7b56fc90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:30:44.409879 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-e86c23f0-61d4-4a33-95e6-f94c7b56fc90 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:30:44.411101 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-e86c23f0-61d4-4a33-95e6-f94c7b56fc90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:30:44.524469 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 653/1327] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:30:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:30:44.805458 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf46c5a0-4f57-4631-bad3-adbcc4559784 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:30:44.807044 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-cf46c5a0-4f57-4631-bad3-adbcc4559784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:30:44.868232 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:30:44.912803 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf46c5a0-4f57-4631-bad3-adbcc4559784 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:30:44.990629 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0bdb18-cb50-4c10-a59a-db6c75c455be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:30:44.991332 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0bdb18-cb50-4c10-a59a-db6c75c455be service neutron] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:30:44.991611 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0bdb18-cb50-4c10-a59a-db6c75c455be service neutron] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:30:44.991881 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0bdb18-cb50-4c10-a59a-db6c75c455be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:30:44.993470 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-7a0bdb18-cb50-4c10-a59a-db6c75c455be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:30:45.100689 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:30:45.103184 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0bdb18-cb50-4c10-a59a-db6c75c455be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:30:45.104613 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0bdb18-cb50-4c10-a59a-db6c75c455be service neutron] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:30:45.124648 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 675/1328] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:30:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:30:47.426892 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb0be0c-83aa-4231-88e2-dce721d5b5dc None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:30:47.427412 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb0be0c-83aa-4231-88e2-dce721d5b5dc None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:30:47.427503 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb0be0c-83aa-4231-88e2-dce721d5b5dc None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:30:47.428798 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 654/1329] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:30:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:30:47.434088 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43054186-8b47-4823-8bf5-7b1544774102 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:30:47.434503 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43054186-8b47-4823-8bf5-7b1544774102 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:30:47.434636 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43054186-8b47-4823-8bf5-7b1544774102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:30:47.501712 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-43054186-8b47-4823-8bf5-7b1544774102 None None] MFA Rules not processed for user `001806a30226487ab9e17f8d988f1ca6`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:30:47.502854 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-43054186-8b47-4823-8bf5-7b1544774102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:30:47.600052 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 676/1330] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:30:47 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:30:47.889699 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb0be0c-83aa-4231-88e2-dce721d5b5dc None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:30:47.891240 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-feb0be0c-83aa-4231-88e2-dce721d5b5dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:30:47.925063 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:30:48.009116 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb0be0c-83aa-4231-88e2-dce721d5b5dc None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:30:48.085882 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e2ebe6-38d4-42da-b16b-57a0a96d2f0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:30:48.086490 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e2ebe6-38d4-42da-b16b-57a0a96d2f0b service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:30:48.086762 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e2ebe6-38d4-42da-b16b-57a0a96d2f0b service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:30:48.087030 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e2ebe6-38d4-42da-b16b-57a0a96d2f0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:30:48.088599 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-08e2ebe6-38d4-42da-b16b-57a0a96d2f0b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:30:48.172030 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:30:48.212526 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e2ebe6-38d4-42da-b16b-57a0a96d2f0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:30:48.214001 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e2ebe6-38d4-42da-b16b-57a0a96d2f0b service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:30:48.234812 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 655/1331] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:30:47 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:30:50.864964 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8436a210-2ed2-4e73-bf1e-a4845d930639 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:30:50.865533 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8436a210-2ed2-4e73-bf1e-a4845d930639 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:30:50.865533 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8436a210-2ed2-4e73-bf1e-a4845d930639 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:30:50.866878 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 677/1332] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:30:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:30:50.872072 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a924c76-e719-43f7-8039-fe12bee094c3 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:30:50.872511 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a924c76-e719-43f7-8039-fe12bee094c3 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:30:50.872622 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a924c76-e719-43f7-8039-fe12bee094c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:30:50.937854 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-9a924c76-e719-43f7-8039-fe12bee094c3 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:30:50.939052 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-9a924c76-e719-43f7-8039-fe12bee094c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:30:51.035823 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 656/1333] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:30:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:30:51.329503 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8436a210-2ed2-4e73-bf1e-a4845d930639 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:30:51.331043 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-8436a210-2ed2-4e73-bf1e-a4845d930639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:30:51.401227 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:30:51.435601 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8436a210-2ed2-4e73-bf1e-a4845d930639 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:30:51.508197 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90585bb-592a-4593-a992-b39212e3b89b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:30:51.508716 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90585bb-592a-4593-a992-b39212e3b89b service neutron] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:30:51.508992 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90585bb-592a-4593-a992-b39212e3b89b service neutron] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:30:51.509262 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90585bb-592a-4593-a992-b39212e3b89b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:30:51.510838 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-b90585bb-592a-4593-a992-b39212e3b89b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:30:51.616278 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90585bb-592a-4593-a992-b39212e3b89b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:30:51.617733 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90585bb-592a-4593-a992-b39212e3b89b service neutron] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:30:51.627339 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:30:51.635172 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 678/1334] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:30:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:30:53.832088 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4909fb-e82b-49a5-a64e-8f5f0f646954 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:30:53.832648 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4909fb-e82b-49a5-a64e-8f5f0f646954 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:30:53.832754 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4909fb-e82b-49a5-a64e-8f5f0f646954 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:30:53.834230 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 657/1335] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:30:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:30:53.839539 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e0681a-3095-4235-bf5b-b1c485822dd5 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:30:53.839918 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e0681a-3095-4235-bf5b-b1c485822dd5 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:30:53.840098 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e0681a-3095-4235-bf5b-b1c485822dd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:30:53.907323 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-69e0681a-3095-4235-bf5b-b1c485822dd5 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:30:53.908528 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-69e0681a-3095-4235-bf5b-b1c485822dd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:30:54.006625 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 679/1336] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:30:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:30:54.290816 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4909fb-e82b-49a5-a64e-8f5f0f646954 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:30:54.292362 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-bb4909fb-e82b-49a5-a64e-8f5f0f646954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:30:54.332166 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:30:54.394109 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4909fb-e82b-49a5-a64e-8f5f0f646954 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:30:54.466922 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8538f50a-6d68-467a-87c7-d9bfdcdde7ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:30:54.467525 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8538f50a-6d68-467a-87c7-d9bfdcdde7ba service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:30:54.467808 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8538f50a-6d68-467a-87c7-d9bfdcdde7ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:30:54.468084 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8538f50a-6d68-467a-87c7-d9bfdcdde7ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:30:54.469690 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-8538f50a-6d68-467a-87c7-d9bfdcdde7ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:30:54.551576 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:30:54.580424 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8538f50a-6d68-467a-87c7-d9bfdcdde7ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:30:54.582436 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8538f50a-6d68-467a-87c7-d9bfdcdde7ba service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:30:54.607772 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 658/1337] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:30:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:30:57.289423 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4561cd2b-2ad8-422d-ba74-ecdd2b4a31c2 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:30:57.290167 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4561cd2b-2ad8-422d-ba74-ecdd2b4a31c2 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:30:57.290167 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4561cd2b-2ad8-422d-ba74-ecdd2b4a31c2 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:30:57.291348 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 680/1338] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:30:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:30:57.296518 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3362abaf-a51a-450c-b748-6dc0248c28e1 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:30:57.296964 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3362abaf-a51a-450c-b748-6dc0248c28e1 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:30:57.297074 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3362abaf-a51a-450c-b748-6dc0248c28e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:30:57.368075 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-3362abaf-a51a-450c-b748-6dc0248c28e1 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:30:57.369274 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-3362abaf-a51a-450c-b748-6dc0248c28e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:30:57.465988 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 659/1339] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:30:57 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:30:57.759074 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4561cd2b-2ad8-422d-ba74-ecdd2b4a31c2 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:30:57.760578 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-4561cd2b-2ad8-422d-ba74-ecdd2b4a31c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:30:57.835226 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:30:57.863545 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4561cd2b-2ad8-422d-ba74-ecdd2b4a31c2 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:30:57.938633 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebc6e95-f50a-4b53-8827-987155fa8b38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:30:57.939156 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebc6e95-f50a-4b53-8827-987155fa8b38 service neutron] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:30:57.939430 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebc6e95-f50a-4b53-8827-987155fa8b38 service neutron] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:30:57.939699 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebc6e95-f50a-4b53-8827-987155fa8b38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:30:57.941349 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-5ebc6e95-f50a-4b53-8827-987155fa8b38 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:30:58.048615 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebc6e95-f50a-4b53-8827-987155fa8b38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:30:58.050084 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebc6e95-f50a-4b53-8827-987155fa8b38 service neutron] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:30:58.068791 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 681/1340] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:30:57 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:31:02.633465 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a702dd-beba-4d55-b8f8-81d98cb65914 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:31:02.634051 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a702dd-beba-4d55-b8f8-81d98cb65914 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:31:02.634051 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a702dd-beba-4d55-b8f8-81d98cb65914 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:31:02.635538 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 660/1341] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:31:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:31:02.640820 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0727fa-286a-47ba-b91e-36f84ec414ee None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:31:02.641277 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0727fa-286a-47ba-b91e-36f84ec414ee None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:31:02.641358 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0727fa-286a-47ba-b91e-36f84ec414ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:31:02.647453 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:31:02.711742 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-ba0727fa-286a-47ba-b91e-36f84ec414ee None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:31:02.712894 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-ba0727fa-286a-47ba-b91e-36f84ec414ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:31:02.810185 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 682/1342] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:31:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:31:03.095915 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a702dd-beba-4d55-b8f8-81d98cb65914 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:31:03.097494 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-59a702dd-beba-4d55-b8f8-81d98cb65914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:31:03.148235 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:31:03.205279 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a702dd-beba-4d55-b8f8-81d98cb65914 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:31:03.281171 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7448b8e-a0b4-4c3f-9942-896db9df0178 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:31:03.281945 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7448b8e-a0b4-4c3f-9942-896db9df0178 service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:31:03.282227 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7448b8e-a0b4-4c3f-9942-896db9df0178 service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:31:03.282499 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7448b8e-a0b4-4c3f-9942-896db9df0178 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:31:03.284087 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-a7448b8e-a0b4-4c3f-9942-896db9df0178 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:31:03.379714 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:31:03.397156 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7448b8e-a0b4-4c3f-9942-896db9df0178 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:31:03.398608 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7448b8e-a0b4-4c3f-9942-896db9df0178 service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:31:03.416729 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 661/1343] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:31:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:31:06.508515 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077f66ef-e66d-4f0f-9236-ce05506cb97e None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:31:06.509091 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077f66ef-e66d-4f0f-9236-ce05506cb97e None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:31:06.509091 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077f66ef-e66d-4f0f-9236-ce05506cb97e None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:31:06.510452 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 683/1344] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:31:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:31:06.515592 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13c0a19-b09c-4d7b-8a30-8c504fe60192 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:31:06.516039 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13c0a19-b09c-4d7b-8a30-8c504fe60192 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:31:06.516144 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13c0a19-b09c-4d7b-8a30-8c504fe60192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:31:06.586715 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-c13c0a19-b09c-4d7b-8a30-8c504fe60192 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:31:06.587977 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-c13c0a19-b09c-4d7b-8a30-8c504fe60192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:31:06.693376 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 662/1345] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:31:06 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:31:06.977730 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077f66ef-e66d-4f0f-9236-ce05506cb97e None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:31:06.979323 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-077f66ef-e66d-4f0f-9236-ce05506cb97e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:31:07.069333 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:31:07.085987 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077f66ef-e66d-4f0f-9236-ce05506cb97e None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:31:07.164476 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a42d92-bde1-4d47-9c12-80bb42177426 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:31:07.165077 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a42d92-bde1-4d47-9c12-80bb42177426 service neutron] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:31:07.165352 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a42d92-bde1-4d47-9c12-80bb42177426 service neutron] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:31:07.165640 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a42d92-bde1-4d47-9c12-80bb42177426 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:31:07.167258 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-54a42d92-bde1-4d47-9c12-80bb42177426 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:31:07.277914 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a42d92-bde1-4d47-9c12-80bb42177426 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:31:07.279378 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a42d92-bde1-4d47-9c12-80bb42177426 service neutron] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:31:07.312905 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 684/1346] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:31:06 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 07:31:11.805247 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca41aac7-d50e-430e-9a6f-b8140c1bf5e8 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:31:11.805787 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca41aac7-d50e-430e-9a6f-b8140c1bf5e8 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:31:11.805868 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca41aac7-d50e-430e-9a6f-b8140c1bf5e8 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:31:11.807377 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 663/1347] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:31:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:31:11.812739 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b3aa6c-38db-4918-ab1e-e31ca1e2d01f None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:31:11.813211 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b3aa6c-38db-4918-ab1e-e31ca1e2d01f None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:31:11.813306 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b3aa6c-38db-4918-ab1e-e31ca1e2d01f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:31:11.829873 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:31:11.882151 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-26b3aa6c-38db-4918-ab1e-e31ca1e2d01f None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:31:11.883351 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-26b3aa6c-38db-4918-ab1e-e31ca1e2d01f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:31:11.988926 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 685/1348] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:31:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:31:12.277061 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca41aac7-d50e-430e-9a6f-b8140c1bf5e8 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:31:12.278629 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-ca41aac7-d50e-430e-9a6f-b8140c1bf5e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:31:12.337089 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:31:12.383394 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca41aac7-d50e-430e-9a6f-b8140c1bf5e8 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:31:12.455430 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcade26d-54dd-49d0-9236-9f7fc80c5f8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:31:12.456033 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcade26d-54dd-49d0-9236-9f7fc80c5f8b service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:31:12.456314 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcade26d-54dd-49d0-9236-9f7fc80c5f8b service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:31:12.456586 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcade26d-54dd-49d0-9236-9f7fc80c5f8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:31:12.458198 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-dcade26d-54dd-49d0-9236-9f7fc80c5f8b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:31:12.564406 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:31:12.568144 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcade26d-54dd-49d0-9236-9f7fc80c5f8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:31:12.569559 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcade26d-54dd-49d0-9236-9f7fc80c5f8b service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:31:12.588853 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 664/1349] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:31:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:31:17.542869 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d7e57a-9629-439c-993f-ce1c621694d0 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:31:17.543401 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d7e57a-9629-439c-993f-ce1c621694d0 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:31:17.543401 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d7e57a-9629-439c-993f-ce1c621694d0 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:31:17.544902 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 686/1350] 10.222.0.42 () {54 vars in 921 bytes} [Thu Sep 15 07:31:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:31:17.552966 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7f2447-0525-4d2c-9412-81512bb6bd5c None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:31:17.552966 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7f2447-0525-4d2c-9412-81512bb6bd5c None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:31:17.552966 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7f2447-0525-4d2c-9412-81512bb6bd5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:31:17.649113 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-ee7f2447-0525-4d2c-9412-81512bb6bd5c None None] MFA Rules not processed for user `dd87846b9a134248b28e0521b0a880c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:31:17.650459 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-ee7f2447-0525-4d2c-9412-81512bb6bd5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:31:17.817089 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 665/1351] 10.222.0.42 () {58 vars in 1018 bytes} [Thu Sep 15 07:31:17 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:31:21.985964 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=59 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:31:22.200336 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082044db-326a-4b88-b17a-149ef4b55ab1 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:31:22.200529 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082044db-326a-4b88-b17a-149ef4b55ab1 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:31:22.200837 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082044db-326a-4b88-b17a-149ef4b55ab1 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:31:22.202467 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 687/1352] 10.222.0.42 () {54 vars in 921 bytes} [Thu Sep 15 07:31:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:31:22.207704 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1382aabb-fdcb-459c-ab85-40e3c760c85a None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:31:22.208174 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1382aabb-fdcb-459c-ab85-40e3c760c85a None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:31:22.208462 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1382aabb-fdcb-459c-ab85-40e3c760c85a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:31:22.267031 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:31:22.290626 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-1382aabb-fdcb-459c-ab85-40e3c760c85a None None] MFA Rules not processed for user `dd87846b9a134248b28e0521b0a880c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:31:22.291876 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-1382aabb-fdcb-459c-ab85-40e3c760c85a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:31:22.433864 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 666/1353] 10.222.0.42 () {58 vars in 1018 bytes} [Thu Sep 15 07:31:22 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:31:23.810153 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a531862a-30a4-4e2e-b3f1-98e5954d52c4 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:31:23.810679 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a531862a-30a4-4e2e-b3f1-98e5954d52c4 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:31:23.810769 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a531862a-30a4-4e2e-b3f1-98e5954d52c4 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:31:23.812061 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 688/1354] 10.222.0.42 () {54 vars in 921 bytes} [Thu Sep 15 07:31:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:31:23.816990 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ee06a-4f5f-4b61-8850-780b33268697 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:31:23.817411 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ee06a-4f5f-4b61-8850-780b33268697 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:31:23.817563 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ee06a-4f5f-4b61-8850-780b33268697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:31:23.895204 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-302ee06a-4f5f-4b61-8850-780b33268697 None None] MFA Rules not processed for user `dd87846b9a134248b28e0521b0a880c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:31:23.897428 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-302ee06a-4f5f-4b61-8850-780b33268697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:31:23.942154 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:31:24.015956 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 667/1355] 10.222.0.42 () {58 vars in 1018 bytes} [Thu Sep 15 07:31:23 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:31:27.519054 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463a0256-8501-45b3-b3af-dc33a1d9a96b None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:31:27.519906 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463a0256-8501-45b3-b3af-dc33a1d9a96b None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:31:27.520174 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463a0256-8501-45b3-b3af-dc33a1d9a96b None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:31:27.521885 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 689/1356] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:31:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:31:27.528057 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836a8a66-c88f-463d-97af-9e393adf90b9 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:31:27.529108 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836a8a66-c88f-463d-97af-9e393adf90b9 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:31:27.529730 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836a8a66-c88f-463d-97af-9e393adf90b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:31:27.627564 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-836a8a66-c88f-463d-97af-9e393adf90b9 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:31:27.629668 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-836a8a66-c88f-463d-97af-9e393adf90b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:31:27.746658 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 668/1357] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:31:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:31:28.093490 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dc7f15-277f-4f4d-9038-6767beab0fcc None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:31:28.093490 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dc7f15-277f-4f4d-9038-6767beab0fcc None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:31:28.093490 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dc7f15-277f-4f4d-9038-6767beab0fcc None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:31:28.093490 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 690/1358] 10.222.0.42 () {54 vars in 962 bytes} [Thu Sep 15 07:31:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:31:28.098814 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f608f16d-effc-4be7-93b2-2d7ab608185f None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:31:28.099199 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f608f16d-effc-4be7-93b2-2d7ab608185f None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:31:28.099969 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f608f16d-effc-4be7-93b2-2d7ab608185f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:31:28.119387 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:31:28.183304 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-f608f16d-effc-4be7-93b2-2d7ab608185f None None] MFA Rules not processed for user `a080c7ab9b684abca6828067f2572c3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:31:28.184503 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-f608f16d-effc-4be7-93b2-2d7ab608185f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:31:28.317756 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 669/1359] 10.222.0.42 () {58 vars in 1059 bytes} [Thu Sep 15 07:31:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 07:31:28.320371 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dc7f15-277f-4f4d-9038-6767beab0fcc None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:31:28.322114 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-a2dc7f15-277f-4f4d-9038-6767beab0fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:31:28.488465 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dc7f15-277f-4f4d-9038-6767beab0fcc None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:31:28.564990 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:31:28.609703 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2d602b-c46f-4461-a5eb-a04bdcc917c4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a080c7ab9b684abca6828067f2572c3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:31:28.610802 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2d602b-c46f-4461-a5eb-a04bdcc917c4 service nova] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:31:28.611082 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2d602b-c46f-4461-a5eb-a04bdcc917c4 service nova] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:31:28.611352 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2d602b-c46f-4461-a5eb-a04bdcc917c4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:31:28.612915 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-ab2d602b-c46f-4461-a5eb-a04bdcc917c4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:31:28.719175 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2d602b-c46f-4461-a5eb-a04bdcc917c4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:31:28.720816 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2d602b-c46f-4461-a5eb-a04bdcc917c4 service nova] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:31:28.741464 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 691/1360] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:31:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 07:31:31.194675 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89503b3a-3464-49a2-a646-84f5c5395697 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:31:31.195164 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89503b3a-3464-49a2-a646-84f5c5395697 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:31:31.195164 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89503b3a-3464-49a2-a646-84f5c5395697 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:31:31.196623 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 670/1361] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:31:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:31:31.201708 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c6c07-5c07-46e3-a3dd-ed7c5bf11b0c None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:31:31.202149 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c6c07-5c07-46e3-a3dd-ed7c5bf11b0c None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:31:31.202302 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c6c07-5c07-46e3-a3dd-ed7c5bf11b0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:31:31.277057 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-0a8c6c07-5c07-46e3-a3dd-ed7c5bf11b0c None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:31:31.278537 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-0a8c6c07-5c07-46e3-a3dd-ed7c5bf11b0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:31:31.322129 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:31:31.386307 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 692/1362] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:31:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:31:31.675289 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89503b3a-3464-49a2-a646-84f5c5395697 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:31:31.676805 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-89503b3a-3464-49a2-a646-84f5c5395697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:31:31.755846 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:31:31.854516 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89503b3a-3464-49a2-a646-84f5c5395697 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:31:31.985698 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c40085-d61b-450c-929b-a12d3a3e6322 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a080c7ab9b684abca6828067f2572c3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:31:31.986313 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c40085-d61b-450c-929b-a12d3a3e6322 service nova] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:31:31.986595 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c40085-d61b-450c-929b-a12d3a3e6322 service nova] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:31:31.987698 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c40085-d61b-450c-929b-a12d3a3e6322 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:31:31.988478 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-87c40085-d61b-450c-929b-a12d3a3e6322 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:31:32.013273 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:31:32.083171 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c40085-d61b-450c-929b-a12d3a3e6322 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:31:32.085390 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c40085-d61b-450c-929b-a12d3a3e6322 service nova] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:31:32.103851 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 671/1363] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:31:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:31:34.762628 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75164b44-d619-4d49-8193-5f6d41320f06 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:31:34.763233 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75164b44-d619-4d49-8193-5f6d41320f06 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:31:34.763233 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75164b44-d619-4d49-8193-5f6d41320f06 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:31:34.764665 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 693/1364] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:31:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:31:34.769793 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc5aa96-158e-4f72-af3d-59cebf58dbb2 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:31:34.770229 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc5aa96-158e-4f72-af3d-59cebf58dbb2 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:31:34.770408 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc5aa96-158e-4f72-af3d-59cebf58dbb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:31:34.839026 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-edc5aa96-158e-4f72-af3d-59cebf58dbb2 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:31:34.840380 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-edc5aa96-158e-4f72-af3d-59cebf58dbb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:31:34.947032 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:31:34.951943 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 672/1365] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:31:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:31:35.241526 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d598495-6510-4a93-87e8-d070a37a17fb None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:31:35.241972 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d598495-6510-4a93-87e8-d070a37a17fb None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:31:35.242192 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d598495-6510-4a93-87e8-d070a37a17fb None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:31:35.243755 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 694/1366] 10.222.0.42 () {54 vars in 962 bytes} [Thu Sep 15 07:31:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:31:35.249622 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d1f31a-4cb3-4d1c-ae14-baea9ddb9184 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:31:35.250077 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d1f31a-4cb3-4d1c-ae14-baea9ddb9184 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:31:35.250416 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d1f31a-4cb3-4d1c-ae14-baea9ddb9184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:31:35.334316 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-90d1f31a-4cb3-4d1c-ae14-baea9ddb9184 None None] MFA Rules not processed for user `a080c7ab9b684abca6828067f2572c3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:31:35.335555 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-90d1f31a-4cb3-4d1c-ae14-baea9ddb9184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:31:35.462400 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 673/1367] 10.222.0.42 () {58 vars in 1059 bytes} [Thu Sep 15 07:31:35 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:31:35.470957 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d598495-6510-4a93-87e8-d070a37a17fb None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:31:35.472432 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-5d598495-6510-4a93-87e8-d070a37a17fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:31:35.633726 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d598495-6510-4a93-87e8-d070a37a17fb None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:31:35.655396 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:31:35.763485 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c48c599-b6c7-425f-9def-3fa6d5132741 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a080c7ab9b684abca6828067f2572c3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:31:35.764178 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c48c599-b6c7-425f-9def-3fa6d5132741 service nova] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:31:35.764448 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c48c599-b6c7-425f-9def-3fa6d5132741 service nova] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:31:35.764631 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c48c599-b6c7-425f-9def-3fa6d5132741 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:31:35.766229 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-2c48c599-b6c7-425f-9def-3fa6d5132741 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:31:35.800595 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e93ad-5ad3-4b19-a414-5772083bce10 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:31:35.801155 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e93ad-5ad3-4b19-a414-5772083bce10 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:31:35.801424 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e93ad-5ad3-4b19-a414-5772083bce10 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:31:35.803009 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 674/1368] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:31:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 07:31:35.808648 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc3f45c-3515-4827-910a-e4fa6006c5a1 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:31:35.809069 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc3f45c-3515-4827-910a-e4fa6006c5a1 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:31:35.809561 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc3f45c-3515-4827-910a-e4fa6006c5a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:31:35.866562 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c48c599-b6c7-425f-9def-3fa6d5132741 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:31:35.868329 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c48c599-b6c7-425f-9def-3fa6d5132741 service nova] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:31:35.872486 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:31:35.879670 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-dfc3f45c-3515-4827-910a-e4fa6006c5a1 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:31:35.880874 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-dfc3f45c-3515-4827-910a-e4fa6006c5a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:31:35.889529 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 695/1369] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:31:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:31:35.997182 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 675/1370] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:31:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 07:31:36.340010 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbc3bba-a44a-4ede-9361-26b33d91e848 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:31:36.340010 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbc3bba-a44a-4ede-9361-26b33d91e848 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:31:36.340010 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbc3bba-a44a-4ede-9361-26b33d91e848 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:31:36.349098 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 696/1371] 10.222.0.42 () {54 vars in 964 bytes} [Thu Sep 15 07:31:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:31:36.350123 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c30f936-c798-435b-947c-3b16ade35342 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:31:36.350123 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c30f936-c798-435b-947c-3b16ade35342 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:31:36.350123 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c30f936-c798-435b-947c-3b16ade35342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:31:36.432593 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-1c30f936-c798-435b-947c-3b16ade35342 None None] MFA Rules not processed for user `3ecf6642b7b84d3c9756bb41ab3349d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:31:36.434072 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-1c30f936-c798-435b-947c-3b16ade35342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:31:36.513049 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:31:36.524254 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 676/1372] 10.222.0.42 () {58 vars in 1061 bytes} [Thu Sep 15 07:31:36 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:31:36.529615 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbc3bba-a44a-4ede-9361-26b33d91e848 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:31:36.531188 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-2bbc3bba-a44a-4ede-9361-26b33d91e848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:31:36.563561 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:31:36.652811 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbc3bba-a44a-4ede-9361-26b33d91e848 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:31:36.741080 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b1aa46-fb26-4766-aea4-e13f8ab1fb15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecf6642b7b84d3c9756bb41ab3349d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:31:36.742019 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b1aa46-fb26-4766-aea4-e13f8ab1fb15 service cinder] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:31:36.742297 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b1aa46-fb26-4766-aea4-e13f8ab1fb15 service cinder] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:31:36.742570 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b1aa46-fb26-4766-aea4-e13f8ab1fb15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:31:36.744429 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-27b1aa46-fb26-4766-aea4-e13f8ab1fb15 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:31:36.826757 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:31:36.869222 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b1aa46-fb26-4766-aea4-e13f8ab1fb15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:31:36.869222 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b1aa46-fb26-4766-aea4-e13f8ab1fb15 service cinder] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:31:36.886805 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 697/1373] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:31:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:31:38.299647 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3546710-9996-49a1-8e52-fc601a25eec1 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:31:38.301440 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3546710-9996-49a1-8e52-fc601a25eec1 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:31:38.301440 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3546710-9996-49a1-8e52-fc601a25eec1 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:31:38.301924 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 677/1374] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:31:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:31:38.307639 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f458c1-e9a9-47e6-8ba9-806ab6711077 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:31:38.308046 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f458c1-e9a9-47e6-8ba9-806ab6711077 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:31:38.308195 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f458c1-e9a9-47e6-8ba9-806ab6711077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:31:38.384439 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-25f458c1-e9a9-47e6-8ba9-806ab6711077 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:31:38.385672 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-25f458c1-e9a9-47e6-8ba9-806ab6711077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:31:38.483310 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 698/1375] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:31:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:31:38.776017 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3546710-9996-49a1-8e52-fc601a25eec1 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:31:38.777539 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-f3546710-9996-49a1-8e52-fc601a25eec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:31:38.932917 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3546710-9996-49a1-8e52-fc601a25eec1 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:31:39.009933 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:31:39.059652 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7448b062-f6f3-42e4-8e75-47ae41d20723 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a080c7ab9b684abca6828067f2572c3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:31:39.060255 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7448b062-f6f3-42e4-8e75-47ae41d20723 service nova] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:31:39.060532 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7448b062-f6f3-42e4-8e75-47ae41d20723 service nova] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:31:39.060800 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7448b062-f6f3-42e4-8e75-47ae41d20723 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:31:39.062878 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-7448b062-f6f3-42e4-8e75-47ae41d20723 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:31:39.158177 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7448b062-f6f3-42e4-8e75-47ae41d20723 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:31:39.159783 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7448b062-f6f3-42e4-8e75-47ae41d20723 service nova] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:31:39.179015 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 678/1376] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:31:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:31:41.578622 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16d1846-24b5-4e26-9ff0-86a11c151650 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:31:41.579167 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16d1846-24b5-4e26-9ff0-86a11c151650 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:31:41.579167 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16d1846-24b5-4e26-9ff0-86a11c151650 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:31:41.580655 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 699/1377] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:31:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:31:41.586022 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334b99f1-3706-41d9-98ce-82fe7b18dab4 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:31:41.586449 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334b99f1-3706-41d9-98ce-82fe7b18dab4 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:31:41.586799 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334b99f1-3706-41d9-98ce-82fe7b18dab4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:31:41.659256 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-334b99f1-3706-41d9-98ce-82fe7b18dab4 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:31:41.660596 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-334b99f1-3706-41d9-98ce-82fe7b18dab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:31:41.695540 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:31:41.773628 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 679/1378] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:31:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:31:42.067278 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16d1846-24b5-4e26-9ff0-86a11c151650 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:31:42.068774 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-f16d1846-24b5-4e26-9ff0-86a11c151650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:31:42.109175 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:31:42.227846 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16d1846-24b5-4e26-9ff0-86a11c151650 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:31:42.345337 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:31:42.351230 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25ae16c-c0e2-44f5-955e-1f2e3bc6032c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a080c7ab9b684abca6828067f2572c3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:31:42.351885 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25ae16c-c0e2-44f5-955e-1f2e3bc6032c service nova] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:31:42.352234 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25ae16c-c0e2-44f5-955e-1f2e3bc6032c service nova] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:31:42.352516 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25ae16c-c0e2-44f5-955e-1f2e3bc6032c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:31:42.354116 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-f25ae16c-c0e2-44f5-955e-1f2e3bc6032c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:31:42.452900 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25ae16c-c0e2-44f5-955e-1f2e3bc6032c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:31:42.454587 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25ae16c-c0e2-44f5-955e-1f2e3bc6032c service nova] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:31:42.474461 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 700/1379] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:31:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:31:44.895442 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451bf4e7-9049-4353-b929-edc1e9f292fa None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:31:44.895959 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451bf4e7-9049-4353-b929-edc1e9f292fa None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:31:44.895959 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451bf4e7-9049-4353-b929-edc1e9f292fa None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:31:44.897283 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 680/1380] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:31:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:31:44.904333 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6dd574-4428-4c44-ba40-781ed0da65e8 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:31:44.904770 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6dd574-4428-4c44-ba40-781ed0da65e8 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:31:44.904921 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6dd574-4428-4c44-ba40-781ed0da65e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:31:44.977932 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-4e6dd574-4428-4c44-ba40-781ed0da65e8 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:31:44.979180 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-4e6dd574-4428-4c44-ba40-781ed0da65e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:31:45.046379 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:31:45.087440 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 701/1381] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:31:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:31:45.465744 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451bf4e7-9049-4353-b929-edc1e9f292fa None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:31:45.467282 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-451bf4e7-9049-4353-b929-edc1e9f292fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:31:45.629825 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451bf4e7-9049-4353-b929-edc1e9f292fa None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:31:45.647454 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:31:45.766445 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4ca3cc-0173-45c4-b765-336429cb9808 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a080c7ab9b684abca6828067f2572c3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:31:45.767060 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4ca3cc-0173-45c4-b765-336429cb9808 service nova] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:31:45.767341 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4ca3cc-0173-45c4-b765-336429cb9808 service nova] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:31:45.767618 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4ca3cc-0173-45c4-b765-336429cb9808 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:31:45.769225 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-7c4ca3cc-0173-45c4-b765-336429cb9808 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:31:45.862010 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4ca3cc-0173-45c4-b765-336429cb9808 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:31:45.863615 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4ca3cc-0173-45c4-b765-336429cb9808 service nova] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:31:45.883244 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 681/1382] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:31:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:31:48.894897 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73776108-beda-49f9-a290-85ccd160e1f5 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:31:48.895788 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73776108-beda-49f9-a290-85ccd160e1f5 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:31:48.895788 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73776108-beda-49f9-a290-85ccd160e1f5 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:31:48.896791 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 702/1383] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:31:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:31:48.902996 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ab6ea0-92f0-47a6-b96c-11d60c1375bf None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:31:48.903444 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ab6ea0-92f0-47a6-b96c-11d60c1375bf None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:31:48.903701 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ab6ea0-92f0-47a6-b96c-11d60c1375bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:31:48.927456 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:31:48.976624 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-f0ab6ea0-92f0-47a6-b96c-11d60c1375bf None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:31:48.977909 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-f0ab6ea0-92f0-47a6-b96c-11d60c1375bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:31:49.079050 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 682/1384] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:31:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:31:49.373662 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73776108-beda-49f9-a290-85ccd160e1f5 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:31:49.375280 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-73776108-beda-49f9-a290-85ccd160e1f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:31:49.560407 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73776108-beda-49f9-a290-85ccd160e1f5 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:31:49.638280 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:31:49.706130 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16f31b2-b3d9-489c-bf30-5e22cffbce93 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a080c7ab9b684abca6828067f2572c3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:31:49.706753 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16f31b2-b3d9-489c-bf30-5e22cffbce93 service nova] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:31:49.707035 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16f31b2-b3d9-489c-bf30-5e22cffbce93 service nova] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:31:49.707309 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16f31b2-b3d9-489c-bf30-5e22cffbce93 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:31:49.708923 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-f16f31b2-b3d9-489c-bf30-5e22cffbce93 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:31:49.816465 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16f31b2-b3d9-489c-bf30-5e22cffbce93 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:31:49.818022 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16f31b2-b3d9-489c-bf30-5e22cffbce93 service nova] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:31:49.838456 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 703/1385] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:31:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 07:31:52.002123 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4ffc18-5b0e-4331-8872-ac4cbb7b12dc None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:31:52.002729 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4ffc18-5b0e-4331-8872-ac4cbb7b12dc None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:31:52.002729 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4ffc18-5b0e-4331-8872-ac4cbb7b12dc None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:31:52.004017 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 683/1386] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:31:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:31:52.009601 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ace508b-9d07-4272-9af5-a8dec53bb911 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:31:52.010062 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ace508b-9d07-4272-9af5-a8dec53bb911 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:31:52.010175 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ace508b-9d07-4272-9af5-a8dec53bb911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:31:52.080650 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-2ace508b-9d07-4272-9af5-a8dec53bb911 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:31:52.081898 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-2ace508b-9d07-4272-9af5-a8dec53bb911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:31:52.091078 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:31:52.180117 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 704/1387] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:31:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:31:52.479655 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4ffc18-5b0e-4331-8872-ac4cbb7b12dc None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:31:52.481204 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-9e4ffc18-5b0e-4331-8872-ac4cbb7b12dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:31:52.509226 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f63b2b3-820d-4b25-9809-5d57f2fad321 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:31:52.509697 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f63b2b3-820d-4b25-9809-5d57f2fad321 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:31:52.510042 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f63b2b3-820d-4b25-9809-5d57f2fad321 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:31:52.511417 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 705/1388] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:31:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:31:52.516584 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb1d0a-ef48-467d-aaa1-00c62c461b56 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:31:52.516867 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb1d0a-ef48-467d-aaa1-00c62c461b56 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:31:52.517142 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb1d0a-ef48-467d-aaa1-00c62c461b56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:31:52.578427 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:31:52.586280 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-59bb1d0a-ef48-467d-aaa1-00c62c461b56 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:31:52.587607 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-59bb1d0a-ef48-467d-aaa1-00c62c461b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:31:52.633529 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4ffc18-5b0e-4331-8872-ac4cbb7b12dc None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:31:52.678022 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:31:52.690251 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 706/1389] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:31:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 07:31:52.752200 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5def9363-d2e0-403d-9c9e-6302ec0087f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a080c7ab9b684abca6828067f2572c3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:31:52.752727 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5def9363-d2e0-403d-9c9e-6302ec0087f0 service nova] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:31:52.753009 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5def9363-d2e0-403d-9c9e-6302ec0087f0 service nova] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:31:52.753285 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5def9363-d2e0-403d-9c9e-6302ec0087f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:31:52.754914 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-5def9363-d2e0-403d-9c9e-6302ec0087f0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:31:52.817054 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:31:52.846791 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5def9363-d2e0-403d-9c9e-6302ec0087f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:31:52.848990 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5def9363-d2e0-403d-9c9e-6302ec0087f0 service nova] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:31:52.868966 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 684/1390] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:31:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:31:53.001798 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0e5b06-d929-4da9-87fb-88b4594939db None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:31:53.002086 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0e5b06-d929-4da9-87fb-88b4594939db None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:31:53.002413 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0e5b06-d929-4da9-87fb-88b4594939db None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:31:53.003717 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 707/1391] 10.222.0.42 () {54 vars in 962 bytes} [Thu Sep 15 07:31:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:31:53.009067 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb78556-3d8e-4734-b8f2-a53d6c98d0d6 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:31:53.009502 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb78556-3d8e-4734-b8f2-a53d6c98d0d6 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:31:53.009704 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb78556-3d8e-4734-b8f2-a53d6c98d0d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:31:53.087086 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-0fb78556-3d8e-4734-b8f2-a53d6c98d0d6 None None] MFA Rules not processed for user `37438b032a1f46d3bdf7121ad0cffbc3`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:31:53.088353 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-0fb78556-3d8e-4734-b8f2-a53d6c98d0d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:31:53.175851 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 685/1392] 10.222.0.42 () {58 vars in 1059 bytes} [Thu Sep 15 07:31:53 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 07:31:53.185511 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0e5b06-d929-4da9-87fb-88b4594939db None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:31:53.187060 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-9c0e5b06-d929-4da9-87fb-88b4594939db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:31:53.299145 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0e5b06-d929-4da9-87fb-88b4594939db None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:31:53.375388 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a5349a-1526-464e-9ea7-6ffea4094e46 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37438b032a1f46d3bdf7121ad0cffbc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:31:53.375995 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a5349a-1526-464e-9ea7-6ffea4094e46 service glance] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:31:53.376273 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a5349a-1526-464e-9ea7-6ffea4094e46 service glance] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:31:53.376544 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a5349a-1526-464e-9ea7-6ffea4094e46 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:31:53.378134 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-e5a5349a-1526-464e-9ea7-6ffea4094e46 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:31:53.424431 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:31:53.492429 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a5349a-1526-464e-9ea7-6ffea4094e46 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:31:53.493902 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a5349a-1526-464e-9ea7-6ffea4094e46 service glance] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:31:53.512985 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 708/1393] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:31:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:31:54.450311 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6361ed47-ee17-41d0-9cbb-305ea9456efb None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:31:54.450795 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6361ed47-ee17-41d0-9cbb-305ea9456efb None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:31:54.450879 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6361ed47-ee17-41d0-9cbb-305ea9456efb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:31:54.527431 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:31:54.531005 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-6361ed47-ee17-41d0-9cbb-305ea9456efb None None] MFA Rules not processed for user `f9c68c4bd9074d8d9b893755e28c0807`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:31:54.532226 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-6361ed47-ee17-41d0-9cbb-305ea9456efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:31:54.625384 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 686/1394] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Sep 15 07:31:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:31:54.672904 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc98ce6-abd3-4a3e-8235-eeb76d1cdfa7 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:31:54.673384 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc98ce6-abd3-4a3e-8235-eeb76d1cdfa7 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:31:54.673484 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc98ce6-abd3-4a3e-8235-eeb76d1cdfa7 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:31:54.674857 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 709/1395] 10.222.0.42 () {54 vars in 943 bytes} [Thu Sep 15 07:31:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:31:54.681914 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572ca7f5-82f6-4af3-8d94-5b5fdb9fe5ef None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:31:54.682276 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572ca7f5-82f6-4af3-8d94-5b5fdb9fe5ef None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:31:54.682646 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572ca7f5-82f6-4af3-8d94-5b5fdb9fe5ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:31:54.767624 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-572ca7f5-82f6-4af3-8d94-5b5fdb9fe5ef None None] MFA Rules not processed for user `72b90d35087e4d7a80e9417082a1541f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:31:54.768833 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-572ca7f5-82f6-4af3-8d94-5b5fdb9fe5ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:31:54.854066 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:31:54.887789 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 687/1396] 10.222.0.42 () {58 vars in 1040 bytes} [Thu Sep 15 07:31:54 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:31:54.896917 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc98ce6-abd3-4a3e-8235-eeb76d1cdfa7 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:31:54.899184 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-8bc98ce6-abd3-4a3e-8235-eeb76d1cdfa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:31:55.044155 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc98ce6-abd3-4a3e-8235-eeb76d1cdfa7 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:31:55.058202 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:31:55.153281 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8524d0-eef5-424c-b7df-29bf07e00b4c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72b90d35087e4d7a80e9417082a1541f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:31:55.153820 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8524d0-eef5-424c-b7df-29bf07e00b4c service swift] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:31:55.154097 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8524d0-eef5-424c-b7df-29bf07e00b4c service swift] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:31:55.154362 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8524d0-eef5-424c-b7df-29bf07e00b4c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:31:55.155957 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-1d8524d0-eef5-424c-b7df-29bf07e00b4c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:31:55.222008 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af3bd8f-c0b7-436f-add1-a3b602338304 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:31:55.222294 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af3bd8f-c0b7-436f-add1-a3b602338304 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:31:55.222570 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af3bd8f-c0b7-436f-add1-a3b602338304 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:31:55.223909 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 688/1397] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:31:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 07:31:55.228874 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda1ff9d-eba3-402a-a9ec-4411850be342 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:31:55.229154 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda1ff9d-eba3-402a-a9ec-4411850be342 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:31:55.229425 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda1ff9d-eba3-402a-a9ec-4411850be342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:31:55.239783 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8524d0-eef5-424c-b7df-29bf07e00b4c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:31:55.241500 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8524d0-eef5-424c-b7df-29bf07e00b4c service swift] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:31:55.250044 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 710/1398] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Sep 15 07:31:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 07:31:55.301315 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-fda1ff9d-eba3-402a-a9ec-4411850be342 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:31:55.302572 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-fda1ff9d-eba3-402a-a9ec-4411850be342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:31:55.401238 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 689/1399] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:31:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 07:31:55.695544 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8524d0-eef5-424c-b7df-29bf07e00b4c service swift] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:31:55.696818 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-1d8524d0-eef5-424c-b7df-29bf07e00b4c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:31:55.751206 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:31:55.799397 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8524d0-eef5-424c-b7df-29bf07e00b4c service swift] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:31:55.961857 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a84b45f-4f47-47d3-832d-f82bb1bdd539 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a080c7ab9b684abca6828067f2572c3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:31:55.962470 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a84b45f-4f47-47d3-832d-f82bb1bdd539 service nova] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:31:55.962748 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a84b45f-4f47-47d3-832d-f82bb1bdd539 service nova] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:31:55.963020 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a84b45f-4f47-47d3-832d-f82bb1bdd539 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:31:55.964786 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-2a84b45f-4f47-47d3-832d-f82bb1bdd539 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:31:56.037805 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:31:56.066940 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a84b45f-4f47-47d3-832d-f82bb1bdd539 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:31:56.068815 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a84b45f-4f47-47d3-832d-f82bb1bdd539 service nova] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:31:56.090462 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 711/1400] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:31:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:31:58.477480 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8afbb96-bc6a-4a5a-9e21-55c7639d9134 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:31:58.478143 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8afbb96-bc6a-4a5a-9e21-55c7639d9134 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:31:58.478143 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8afbb96-bc6a-4a5a-9e21-55c7639d9134 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:31:58.479432 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 690/1401] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:31:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:31:58.484781 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037d71b8-eba9-45f6-bfb1-4038281ec1d6 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:31:58.485241 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037d71b8-eba9-45f6-bfb1-4038281ec1d6 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:31:58.485544 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037d71b8-eba9-45f6-bfb1-4038281ec1d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:31:58.563075 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-037d71b8-eba9-45f6-bfb1-4038281ec1d6 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:31:58.564574 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-037d71b8-eba9-45f6-bfb1-4038281ec1d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:31:58.675959 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 712/1402] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:31:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:31:58.967480 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8afbb96-bc6a-4a5a-9e21-55c7639d9134 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:31:58.969098 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-b8afbb96-bc6a-4a5a-9e21-55c7639d9134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:31:59.031963 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:31:59.132335 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8afbb96-bc6a-4a5a-9e21-55c7639d9134 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:31:59.255731 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c531de8-0014-4dcd-9cd0-b695aeb2398b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a080c7ab9b684abca6828067f2572c3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:31:59.258605 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c531de8-0014-4dcd-9cd0-b695aeb2398b service nova] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:31:59.258981 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c531de8-0014-4dcd-9cd0-b695aeb2398b service nova] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:31:59.259264 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c531de8-0014-4dcd-9cd0-b695aeb2398b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:31:59.260933 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-5c531de8-0014-4dcd-9cd0-b695aeb2398b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:31:59.277019 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:31:59.365040 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c531de8-0014-4dcd-9cd0-b695aeb2398b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:31:59.367311 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c531de8-0014-4dcd-9cd0-b695aeb2398b service nova] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:31:59.388282 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 691/1403] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:31:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 07:32:01.761974 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed06bac1-b2eb-4fb8-8c6d-76bc33daa07f None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:32:01.762860 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed06bac1-b2eb-4fb8-8c6d-76bc33daa07f None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:32:01.762860 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed06bac1-b2eb-4fb8-8c6d-76bc33daa07f None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:32:01.763889 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 713/1404] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:32:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:32:01.770926 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c138e86c-40a0-4797-a8a4-438dfbf6dc35 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:32:01.771575 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c138e86c-40a0-4797-a8a4-438dfbf6dc35 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:32:01.771575 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c138e86c-40a0-4797-a8a4-438dfbf6dc35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:32:01.861417 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-c138e86c-40a0-4797-a8a4-438dfbf6dc35 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:32:01.862848 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-c138e86c-40a0-4797-a8a4-438dfbf6dc35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:32:01.993041 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:32:02.007664 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 692/1405] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:32:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:32:02.337328 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed06bac1-b2eb-4fb8-8c6d-76bc33daa07f None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:32:02.338914 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-ed06bac1-b2eb-4fb8-8c6d-76bc33daa07f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:32:02.397370 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:32:02.522628 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed06bac1-b2eb-4fb8-8c6d-76bc33daa07f None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:32:02.651854 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949ad9ea-ee64-4a8c-aaa3-277f2fe62b64 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a080c7ab9b684abca6828067f2572c3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:32:02.652564 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949ad9ea-ee64-4a8c-aaa3-277f2fe62b64 service nova] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:32:02.652867 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949ad9ea-ee64-4a8c-aaa3-277f2fe62b64 service nova] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:32:02.653139 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949ad9ea-ee64-4a8c-aaa3-277f2fe62b64 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:32:02.654751 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-949ad9ea-ee64-4a8c-aaa3-277f2fe62b64 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:32:02.658660 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:32:02.766744 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949ad9ea-ee64-4a8c-aaa3-277f2fe62b64 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:32:02.768418 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949ad9ea-ee64-4a8c-aaa3-277f2fe62b64 service nova] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:32:02.788438 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 714/1406] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:32:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:32:05.177401 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb304a36-8039-41f5-8aca-7bf4f2869e6c None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:32:05.177979 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb304a36-8039-41f5-8aca-7bf4f2869e6c None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:32:05.177979 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb304a36-8039-41f5-8aca-7bf4f2869e6c None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:32:05.179347 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 693/1407] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:32:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:32:05.185168 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f209c77-01b7-4f2e-80b3-2682745b23a2 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:32:05.185608 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f209c77-01b7-4f2e-80b3-2682745b23a2 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:32:05.185816 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f209c77-01b7-4f2e-80b3-2682745b23a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:32:05.259118 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-0f209c77-01b7-4f2e-80b3-2682745b23a2 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:32:05.260233 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-0f209c77-01b7-4f2e-80b3-2682745b23a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:32:05.339155 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:32:05.369753 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 715/1408] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:32:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:32:05.661175 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb304a36-8039-41f5-8aca-7bf4f2869e6c None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:32:05.662722 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-bb304a36-8039-41f5-8aca-7bf4f2869e6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:32:05.826207 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb304a36-8039-41f5-8aca-7bf4f2869e6c None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:32:05.897469 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:32:05.957122 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e507388-e334-453f-9a5d-8e964b3f7df9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a080c7ab9b684abca6828067f2572c3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:32:05.957861 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e507388-e334-453f-9a5d-8e964b3f7df9 service nova] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:32:05.958141 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e507388-e334-453f-9a5d-8e964b3f7df9 service nova] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:32:05.958412 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e507388-e334-453f-9a5d-8e964b3f7df9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:32:05.960036 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-6e507388-e334-453f-9a5d-8e964b3f7df9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:32:06.058752 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e507388-e334-453f-9a5d-8e964b3f7df9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:32:06.060370 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e507388-e334-453f-9a5d-8e964b3f7df9 service nova] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:32:06.079534 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 694/1409] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:32:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:32:15.369531 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=9 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:32:16.077083 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=66 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:32:19.506656 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb3d128-cd9e-40de-b924-ad3ff858397f None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:32:19.508427 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb3d128-cd9e-40de-b924-ad3ff858397f None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:32:19.508427 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb3d128-cd9e-40de-b924-ad3ff858397f None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:32:19.509296 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 716/1410] 10.222.0.42 () {54 vars in 918 bytes} [Thu Sep 15 07:32:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:32:19.514386 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a5208a-23d2-4137-83e4-21600ec34087 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:32:19.514823 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a5208a-23d2-4137-83e4-21600ec34087 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:32:19.514944 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a5208a-23d2-4137-83e4-21600ec34087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:32:19.591056 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-c6a5208a-23d2-4137-83e4-21600ec34087 None None] MFA Rules not processed for user `dd87846b9a134248b28e0521b0a880c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:32:19.592279 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-c6a5208a-23d2-4137-83e4-21600ec34087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:32:19.706323 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 695/1411] 10.222.0.42 () {58 vars in 1015 bytes} [Thu Sep 15 07:32:19 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:32:19.721109 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f92339b-1d33-42a6-9145-841611e8a095 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:32:19.721396 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f92339b-1d33-42a6-9145-841611e8a095 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:32:19.721704 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f92339b-1d33-42a6-9145-841611e8a095 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:32:19.723108 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 717/1412] 10.222.0.42 () {54 vars in 918 bytes} [Thu Sep 15 07:32:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:32:19.728774 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d88b8e-7fff-4a63-8fe6-8555df8b1b19 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:32:19.729065 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d88b8e-7fff-4a63-8fe6-8555df8b1b19 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:32:19.729344 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d88b8e-7fff-4a63-8fe6-8555df8b1b19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:32:19.808533 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:32:19.811164 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-12d88b8e-7fff-4a63-8fe6-8555df8b1b19 None None] MFA Rules not processed for user `a080c7ab9b684abca6828067f2572c3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:32:19.812385 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-12d88b8e-7fff-4a63-8fe6-8555df8b1b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:32:19.934259 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 696/1413] 10.222.0.42 () {58 vars in 1015 bytes} [Thu Sep 15 07:32:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:32:22.672552 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738131a6-7a96-4cea-bfb0-080d5d17bd95 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:32:22.673084 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738131a6-7a96-4cea-bfb0-080d5d17bd95 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:32:22.673084 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738131a6-7a96-4cea-bfb0-080d5d17bd95 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:32:22.674369 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 718/1414] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 07:32:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:32:22.679696 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d371c73-3f35-4690-a160-00cac3310474 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:32:22.680136 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d371c73-3f35-4690-a160-00cac3310474 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:32:22.680292 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d371c73-3f35-4690-a160-00cac3310474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:32:22.751595 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-5d371c73-3f35-4690-a160-00cac3310474 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:32:22.752827 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-5d371c73-3f35-4690-a160-00cac3310474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:32:22.813447 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:32:22.855924 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 697/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 07:32:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:32:23.152883 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738131a6-7a96-4cea-bfb0-080d5d17bd95 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:32:23.154411 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-738131a6-7a96-4cea-bfb0-080d5d17bd95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:32:23.304929 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738131a6-7a96-4cea-bfb0-080d5d17bd95 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:32:23.379368 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:32:23.422396 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3921c07-dfec-41dd-9184-097b53c7505c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a080c7ab9b684abca6828067f2572c3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:32:23.423010 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3921c07-dfec-41dd-9184-097b53c7505c service nova] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:32:23.423294 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3921c07-dfec-41dd-9184-097b53c7505c service nova] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:32:23.423571 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3921c07-dfec-41dd-9184-097b53c7505c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:32:23.425156 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-b3921c07-dfec-41dd-9184-097b53c7505c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:32:23.517399 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3921c07-dfec-41dd-9184-097b53c7505c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:32:23.519015 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3921c07-dfec-41dd-9184-097b53c7505c service nova] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:32:23.536736 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 719/1416] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:32:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:32:24.686469 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34288c52-6f26-43cc-b6fd-68b95311e00c None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:32:24.687001 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34288c52-6f26-43cc-b6fd-68b95311e00c None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:32:24.687001 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34288c52-6f26-43cc-b6fd-68b95311e00c None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:32:24.688375 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 698/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 15 07:32:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:32:24.694631 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327c0239-165c-46eb-a3f1-1885da7a3aa8 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:32:24.695082 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327c0239-165c-46eb-a3f1-1885da7a3aa8 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:32:24.695182 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327c0239-165c-46eb-a3f1-1885da7a3aa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:32:24.772516 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-327c0239-165c-46eb-a3f1-1885da7a3aa8 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:32:24.773736 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-327c0239-165c-46eb-a3f1-1885da7a3aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:32:24.800708 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:32:24.870219 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 720/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 15 07:32:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:32:25.021127 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34288c52-6f26-43cc-b6fd-68b95311e00c None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:32:25.022713 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-34288c52-6f26-43cc-b6fd-68b95311e00c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:32:25.170167 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34288c52-6f26-43cc-b6fd-68b95311e00c None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:32:25.326945 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:32:25.400395 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f70320-326c-4afb-bde1-2c070f25600b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a080c7ab9b684abca6828067f2572c3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:32:25.400997 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f70320-326c-4afb-bde1-2c070f25600b service nova] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:32:25.401276 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f70320-326c-4afb-bde1-2c070f25600b service nova] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:32:25.401548 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f70320-326c-4afb-bde1-2c070f25600b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:32:25.403173 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-f8f70320-326c-4afb-bde1-2c070f25600b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:32:25.493433 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f70320-326c-4afb-bde1-2c070f25600b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:32:25.495706 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f70320-326c-4afb-bde1-2c070f25600b service nova] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:32:25.513445 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 699/1419] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:32:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:32:26.666161 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183df6f6-1979-40ce-aa83-1fa0aa5befa4 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:32:26.666744 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183df6f6-1979-40ce-aa83-1fa0aa5befa4 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:32:26.666744 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183df6f6-1979-40ce-aa83-1fa0aa5befa4 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:32:26.668058 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 721/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 15 07:32:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:32:26.674264 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5704324f-8b71-46fb-89bd-6ae401c030c6 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:32:26.674677 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5704324f-8b71-46fb-89bd-6ae401c030c6 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:32:26.674818 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5704324f-8b71-46fb-89bd-6ae401c030c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:32:26.730630 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:32:26.752649 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-5704324f-8b71-46fb-89bd-6ae401c030c6 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:32:26.753883 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-5704324f-8b71-46fb-89bd-6ae401c030c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:32:26.849944 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 700/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 15 07:32:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:32:27.007875 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183df6f6-1979-40ce-aa83-1fa0aa5befa4 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:32:27.009434 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-183df6f6-1979-40ce-aa83-1fa0aa5befa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:32:27.154765 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183df6f6-1979-40ce-aa83-1fa0aa5befa4 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:32:27.168778 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:32:27.276166 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9e120c-2610-4ef6-967e-3e2544d49f34 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a080c7ab9b684abca6828067f2572c3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:32:27.276696 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9e120c-2610-4ef6-967e-3e2544d49f34 service nova] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:32:27.276981 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9e120c-2610-4ef6-967e-3e2544d49f34 service nova] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:32:27.277251 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9e120c-2610-4ef6-967e-3e2544d49f34 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:32:27.278856 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-bc9e120c-2610-4ef6-967e-3e2544d49f34 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:32:27.378974 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9e120c-2610-4ef6-967e-3e2544d49f34 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:32:27.380651 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9e120c-2610-4ef6-967e-3e2544d49f34 service nova] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:32:27.398877 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 722/1422] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:32:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:32:28.524189 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43193f5-635e-4dfc-93f2-76ea9e4f73e6 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:32:28.524806 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43193f5-635e-4dfc-93f2-76ea9e4f73e6 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:32:28.524917 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43193f5-635e-4dfc-93f2-76ea9e4f73e6 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:32:28.526494 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 701/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 15 07:32:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:32:28.532823 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff2ce06-ef50-4441-a154-2b4f43957db3 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:32:28.533231 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff2ce06-ef50-4441-a154-2b4f43957db3 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:32:28.533424 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff2ce06-ef50-4441-a154-2b4f43957db3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:32:28.553010 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:32:28.622114 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-cff2ce06-ef50-4441-a154-2b4f43957db3 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:32:28.623323 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-cff2ce06-ef50-4441-a154-2b4f43957db3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:32:28.724140 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 723/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 15 07:32:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 07:32:28.879795 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43193f5-635e-4dfc-93f2-76ea9e4f73e6 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:32:28.881307 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-d43193f5-635e-4dfc-93f2-76ea9e4f73e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:32:28.995393 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:32:29.029797 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43193f5-635e-4dfc-93f2-76ea9e4f73e6 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:32:29.140121 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72c3722-0a30-4964-907e-79fbc2b94ab0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a080c7ab9b684abca6828067f2572c3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:32:29.140638 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72c3722-0a30-4964-907e-79fbc2b94ab0 service nova] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:32:29.140920 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72c3722-0a30-4964-907e-79fbc2b94ab0 service nova] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:32:29.141185 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72c3722-0a30-4964-907e-79fbc2b94ab0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:32:29.142780 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-f72c3722-0a30-4964-907e-79fbc2b94ab0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:32:29.217699 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:32:29.230431 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72c3722-0a30-4964-907e-79fbc2b94ab0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:32:29.232077 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72c3722-0a30-4964-907e-79fbc2b94ab0 service nova] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:32:29.251283 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 702/1425] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:32:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:32:30.379172 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db7b094-3b71-4c7f-815a-d8bdb5db8149 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:32:30.379735 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db7b094-3b71-4c7f-815a-d8bdb5db8149 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:32:30.379735 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db7b094-3b71-4c7f-815a-d8bdb5db8149 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:32:30.381112 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 724/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 15 07:32:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:32:30.387948 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1b2981-3c68-4bf0-910f-ffde3c338e36 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:32:30.388392 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1b2981-3c68-4bf0-910f-ffde3c338e36 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:32:30.388498 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1b2981-3c68-4bf0-910f-ffde3c338e36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:32:30.464916 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-7a1b2981-3c68-4bf0-910f-ffde3c338e36 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:32:30.466135 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-7a1b2981-3c68-4bf0-910f-ffde3c338e36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:32:30.561133 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 703/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 15 07:32:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:32:30.714056 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db7b094-3b71-4c7f-815a-d8bdb5db8149 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:32:30.715489 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-1db7b094-3b71-4c7f-815a-d8bdb5db8149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:32:30.805975 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:32:30.867524 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db7b094-3b71-4c7f-815a-d8bdb5db8149 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:32:30.979407 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcfd0e5-3c40-4671-9fff-b15448a2f6d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a080c7ab9b684abca6828067f2572c3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:32:30.979933 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcfd0e5-3c40-4671-9fff-b15448a2f6d1 service nova] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:32:30.980212 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcfd0e5-3c40-4671-9fff-b15448a2f6d1 service nova] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:32:30.980483 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcfd0e5-3c40-4671-9fff-b15448a2f6d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:32:30.982089 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-0fcfd0e5-3c40-4671-9fff-b15448a2f6d1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:32:31.030162 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:32:31.069724 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcfd0e5-3c40-4671-9fff-b15448a2f6d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:32:31.071302 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcfd0e5-3c40-4671-9fff-b15448a2f6d1 service nova] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:32:31.089593 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 725/1428] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:32:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:32:32.236626 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9846a1a-4bf6-4378-9823-fdf77b309758 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:32:32.237160 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9846a1a-4bf6-4378-9823-fdf77b309758 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:32:32.237160 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9846a1a-4bf6-4378-9823-fdf77b309758 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:32:32.238516 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 704/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 15 07:32:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:32:32.245701 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf2bcd0-de69-418c-80e1-65b72ca33576 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:32:32.246153 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf2bcd0-de69-418c-80e1-65b72ca33576 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:32:32.246255 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf2bcd0-de69-418c-80e1-65b72ca33576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:32:32.322699 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-bbf2bcd0-de69-418c-80e1-65b72ca33576 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:32:32.323889 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-bbf2bcd0-de69-418c-80e1-65b72ca33576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:32:32.420732 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 726/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 15 07:32:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:32:32.579270 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9846a1a-4bf6-4378-9823-fdf77b309758 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:32:32.580911 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-e9846a1a-4bf6-4378-9823-fdf77b309758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:32:32.630743 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:32:32.726012 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9846a1a-4bf6-4378-9823-fdf77b309758 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:32:32.839044 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b5eab2-6186-44a9-8c01-db53e1135113 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a080c7ab9b684abca6828067f2572c3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:32:32.839561 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b5eab2-6186-44a9-8c01-db53e1135113 service nova] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:32:32.839835 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b5eab2-6186-44a9-8c01-db53e1135113 service nova] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:32:32.840102 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b5eab2-6186-44a9-8c01-db53e1135113 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:32:32.841891 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-d1b5eab2-6186-44a9-8c01-db53e1135113 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:32:32.848647 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:32:32.930056 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b5eab2-6186-44a9-8c01-db53e1135113 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:32:32.932180 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b5eab2-6186-44a9-8c01-db53e1135113 service nova] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:32:32.950484 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 705/1431] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:32:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 07:32:34.056251 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d98eda-7d6a-415b-b52d-a5b482de6250 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:32:34.056786 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d98eda-7d6a-415b-b52d-a5b482de6250 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:32:34.056786 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d98eda-7d6a-415b-b52d-a5b482de6250 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:32:34.058254 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 727/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 15 07:32:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:32:34.064464 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07b9fc4-f8b7-4f4b-a135-16dc163667e9 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:32:34.064919 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07b9fc4-f8b7-4f4b-a135-16dc163667e9 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:32:34.065013 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07b9fc4-f8b7-4f4b-a135-16dc163667e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:32:34.140933 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-c07b9fc4-f8b7-4f4b-a135-16dc163667e9 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:32:34.142179 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-c07b9fc4-f8b7-4f4b-a135-16dc163667e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:32:34.209037 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:32:34.238513 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 706/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 15 07:32:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:32:34.393894 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d98eda-7d6a-415b-b52d-a5b482de6250 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:32:34.395396 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-77d98eda-7d6a-415b-b52d-a5b482de6250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:32:34.413676 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:32:34.541255 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d98eda-7d6a-415b-b52d-a5b482de6250 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:32:34.633837 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:32:34.659597 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f32ac8-7a9e-4635-bc77-6aabd7128ee6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a080c7ab9b684abca6828067f2572c3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:32:34.660190 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f32ac8-7a9e-4635-bc77-6aabd7128ee6 service nova] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:32:34.660463 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f32ac8-7a9e-4635-bc77-6aabd7128ee6 service nova] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:32:34.660731 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f32ac8-7a9e-4635-bc77-6aabd7128ee6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:32:34.662317 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-68f32ac8-7a9e-4635-bc77-6aabd7128ee6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:32:34.752204 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f32ac8-7a9e-4635-bc77-6aabd7128ee6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:32:34.753827 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f32ac8-7a9e-4635-bc77-6aabd7128ee6 service nova] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:32:34.771794 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 728/1434] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:32:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:32:35.889524 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c5a645-fcaf-4be1-b34e-d032c9331b67 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:32:35.890128 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c5a645-fcaf-4be1-b34e-d032c9331b67 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:32:35.890128 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c5a645-fcaf-4be1-b34e-d032c9331b67 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:32:35.891461 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 707/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 15 07:32:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:32:35.897681 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e130ccbd-6864-4a05-b565-74cb4ca73b23 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:32:35.898114 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e130ccbd-6864-4a05-b565-74cb4ca73b23 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:32:35.898234 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e130ccbd-6864-4a05-b565-74cb4ca73b23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:32:35.975517 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-e130ccbd-6864-4a05-b565-74cb4ca73b23 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:32:35.976721 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-e130ccbd-6864-4a05-b565-74cb4ca73b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:32:36.019049 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:32:36.073916 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 729/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 15 07:32:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:32:36.225131 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c5a645-fcaf-4be1-b34e-d032c9331b67 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:32:36.226629 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-49c5a645-fcaf-4be1-b34e-d032c9331b67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:32:36.370550 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c5a645-fcaf-4be1-b34e-d032c9331b67 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:32:36.424257 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:32:36.483981 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2837d4-3daf-40d8-8955-0db04a8ff6af service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a080c7ab9b684abca6828067f2572c3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:32:36.484504 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2837d4-3daf-40d8-8955-0db04a8ff6af service nova] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:32:36.484781 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2837d4-3daf-40d8-8955-0db04a8ff6af service nova] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:32:36.485052 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2837d4-3daf-40d8-8955-0db04a8ff6af service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:32:36.486676 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-2e2837d4-3daf-40d8-8955-0db04a8ff6af service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:32:36.576011 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2837d4-3daf-40d8-8955-0db04a8ff6af service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:32:36.577614 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2837d4-3daf-40d8-8955-0db04a8ff6af service nova] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:32:36.597455 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 708/1437] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:32:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:32:37.722139 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd085c6-a8b7-4642-b48a-dfb1eec8fdc5 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:32:37.722707 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd085c6-a8b7-4642-b48a-dfb1eec8fdc5 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:32:37.722707 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd085c6-a8b7-4642-b48a-dfb1eec8fdc5 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:32:37.724049 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 730/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 15 07:32:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:32:37.731106 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4924f29-4743-4fa4-9ba2-47fbde73eac6 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:32:37.731529 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4924f29-4743-4fa4-9ba2-47fbde73eac6 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:32:37.731668 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4924f29-4743-4fa4-9ba2-47fbde73eac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:32:37.807263 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-e4924f29-4743-4fa4-9ba2-47fbde73eac6 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:32:37.808486 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-e4924f29-4743-4fa4-9ba2-47fbde73eac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:32:37.810566 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:32:37.906343 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 709/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 15 07:32:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:32:38.056922 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd085c6-a8b7-4642-b48a-dfb1eec8fdc5 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:32:38.058486 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-3dd085c6-a8b7-4642-b48a-dfb1eec8fdc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:32:38.209484 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd085c6-a8b7-4642-b48a-dfb1eec8fdc5 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:32:38.239297 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:32:38.338866 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d361b9-6d7f-435f-adae-7d823305c39e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a080c7ab9b684abca6828067f2572c3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:32:38.339470 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d361b9-6d7f-435f-adae-7d823305c39e service nova] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:32:38.339750 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d361b9-6d7f-435f-adae-7d823305c39e service nova] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:32:38.340022 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d361b9-6d7f-435f-adae-7d823305c39e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:32:38.341706 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-e3d361b9-6d7f-435f-adae-7d823305c39e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:32:38.432441 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d361b9-6d7f-435f-adae-7d823305c39e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:32:38.434061 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d361b9-6d7f-435f-adae-7d823305c39e service nova] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:32:38.452240 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 731/1440] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:32:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 07:32:40.541946 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55ffbcd-6b4c-4768-b968-76c6f40e8d3c None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:32:40.542517 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55ffbcd-6b4c-4768-b968-76c6f40e8d3c None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:32:40.542517 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55ffbcd-6b4c-4768-b968-76c6f40e8d3c None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:32:40.543852 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 710/1441] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 07:32:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:32:40.549323 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d6eb2f-85de-48ff-a0f0-8ed9ca177828 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:32:40.549752 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d6eb2f-85de-48ff-a0f0-8ed9ca177828 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:32:40.549922 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d6eb2f-85de-48ff-a0f0-8ed9ca177828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:32:40.591475 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:32:40.620368 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-81d6eb2f-85de-48ff-a0f0-8ed9ca177828 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:32:40.622398 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-81d6eb2f-85de-48ff-a0f0-8ed9ca177828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:32:40.718587 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 732/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 07:32:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:32:41.008880 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55ffbcd-6b4c-4768-b968-76c6f40e8d3c None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:32:41.011234 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-d55ffbcd-6b4c-4768-b968-76c6f40e8d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:32:41.114396 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55ffbcd-6b4c-4768-b968-76c6f40e8d3c None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:32:41.146926 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:32:41.187225 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22129c45-1a6e-4df7-b645-68893f9e7e92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:32:41.187816 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22129c45-1a6e-4df7-b645-68893f9e7e92 service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:32:41.188093 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22129c45-1a6e-4df7-b645-68893f9e7e92 service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:32:41.188358 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22129c45-1a6e-4df7-b645-68893f9e7e92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:32:41.189975 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-22129c45-1a6e-4df7-b645-68893f9e7e92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:32:41.299026 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22129c45-1a6e-4df7-b645-68893f9e7e92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:32:41.300541 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22129c45-1a6e-4df7-b645-68893f9e7e92 service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:32:41.318972 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 711/1443] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:32:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:32:43.635806 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a81e14b-dc4a-432a-98e0-a085b750b480 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:32:43.636341 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a81e14b-dc4a-432a-98e0-a085b750b480 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:32:43.636341 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a81e14b-dc4a-432a-98e0-a085b750b480 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:32:43.637631 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 733/1444] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 07:32:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:32:43.642839 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a79218e-8a48-47db-bef2-2fc3fb0b7def None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:32:43.643284 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a79218e-8a48-47db-bef2-2fc3fb0b7def None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:32:43.643422 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a79218e-8a48-47db-bef2-2fc3fb0b7def None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:32:43.713886 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-2a79218e-8a48-47db-bef2-2fc3fb0b7def None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:32:43.715119 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-2a79218e-8a48-47db-bef2-2fc3fb0b7def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:32:43.727809 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:32:43.813074 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 712/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 07:32:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:32:44.097420 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a81e14b-dc4a-432a-98e0-a085b750b480 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:32:44.099031 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-6a81e14b-dc4a-432a-98e0-a085b750b480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:32:44.202529 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a81e14b-dc4a-432a-98e0-a085b750b480 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:32:44.215597 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:32:44.277898 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1a868f-59fd-4be6-90b2-22cac6dd5974 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:32:44.278510 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1a868f-59fd-4be6-90b2-22cac6dd5974 service neutron] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:32:44.278793 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1a868f-59fd-4be6-90b2-22cac6dd5974 service neutron] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:32:44.279068 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1a868f-59fd-4be6-90b2-22cac6dd5974 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:32:44.280664 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-8b1a868f-59fd-4be6-90b2-22cac6dd5974 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:32:44.393479 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1a868f-59fd-4be6-90b2-22cac6dd5974 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:32:44.395005 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1a868f-59fd-4be6-90b2-22cac6dd5974 service neutron] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:32:44.413937 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 734/1446] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:32:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:32:46.999829 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28a4286-c66c-40f5-a51f-e5b19057f010 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:32:47.000372 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28a4286-c66c-40f5-a51f-e5b19057f010 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:32:47.000598 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28a4286-c66c-40f5-a51f-e5b19057f010 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:32:47.002035 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 713/1447] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 07:32:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:32:47.007680 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287bd69f-ea78-42af-a336-5227ba9387fc None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:32:47.008137 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287bd69f-ea78-42af-a336-5227ba9387fc None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:32:47.008234 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287bd69f-ea78-42af-a336-5227ba9387fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:32:47.056186 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:32:47.074475 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-287bd69f-ea78-42af-a336-5227ba9387fc None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:32:47.076504 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-287bd69f-ea78-42af-a336-5227ba9387fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:32:47.170788 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 735/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 07:32:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:32:47.456942 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28a4286-c66c-40f5-a51f-e5b19057f010 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:32:47.458519 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-a28a4286-c66c-40f5-a51f-e5b19057f010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:32:47.560775 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28a4286-c66c-40f5-a51f-e5b19057f010 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:32:47.604063 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:32:47.642376 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d31f6f-fad1-4149-af36-e450f727d0c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:32:47.643043 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d31f6f-fad1-4149-af36-e450f727d0c3 service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:32:47.643327 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d31f6f-fad1-4149-af36-e450f727d0c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:32:47.643601 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d31f6f-fad1-4149-af36-e450f727d0c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:32:47.645183 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-25d31f6f-fad1-4149-af36-e450f727d0c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:32:47.754598 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d31f6f-fad1-4149-af36-e450f727d0c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:32:47.756020 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d31f6f-fad1-4149-af36-e450f727d0c3 service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:32:47.774681 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 714/1449] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:32:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:32:50.123237 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646404ac-0133-4da4-bd6a-914c9eba7bfe None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:32:50.123810 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646404ac-0133-4da4-bd6a-914c9eba7bfe None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:32:50.123810 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646404ac-0133-4da4-bd6a-914c9eba7bfe None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:32:50.125140 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 736/1450] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 07:32:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:32:50.130498 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d311eb6-2646-4685-9ead-4e968345ee09 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:32:50.130973 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d311eb6-2646-4685-9ead-4e968345ee09 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:32:50.131085 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d311eb6-2646-4685-9ead-4e968345ee09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:32:50.196599 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-4d311eb6-2646-4685-9ead-4e968345ee09 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:32:50.197785 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-4d311eb6-2646-4685-9ead-4e968345ee09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:32:50.218381 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:32:50.295791 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 715/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 07:32:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:32:50.580176 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646404ac-0133-4da4-bd6a-914c9eba7bfe None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:32:50.581727 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-646404ac-0133-4da4-bd6a-914c9eba7bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:32:50.687426 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646404ac-0133-4da4-bd6a-914c9eba7bfe None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:32:50.708135 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:32:50.761382 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062145da-b8cd-4e9e-a784-383ed3a31e48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:32:50.762022 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062145da-b8cd-4e9e-a784-383ed3a31e48 service neutron] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:32:50.762297 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062145da-b8cd-4e9e-a784-383ed3a31e48 service neutron] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:32:50.762567 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062145da-b8cd-4e9e-a784-383ed3a31e48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:32:50.764131 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-062145da-b8cd-4e9e-a784-383ed3a31e48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:32:50.879393 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062145da-b8cd-4e9e-a784-383ed3a31e48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:32:50.880751 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062145da-b8cd-4e9e-a784-383ed3a31e48 service neutron] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:32:50.898877 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 737/1452] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:32:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:32:53.098933 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bcb96c-f81b-4722-ab7a-e35dfedb081c None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:32:53.099492 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bcb96c-f81b-4722-ab7a-e35dfedb081c None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:32:53.099492 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bcb96c-f81b-4722-ab7a-e35dfedb081c None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:32:53.100847 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 716/1453] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 07:32:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:32:53.106906 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338e732a-5cca-4d5e-826c-b881c9df051d None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:32:53.107350 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338e732a-5cca-4d5e-826c-b881c9df051d None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:32:53.107456 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338e732a-5cca-4d5e-826c-b881c9df051d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:32:53.170562 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:32:53.173297 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-338e732a-5cca-4d5e-826c-b881c9df051d None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:32:53.174561 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-338e732a-5cca-4d5e-826c-b881c9df051d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:32:53.269816 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 738/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 07:32:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:32:53.562535 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bcb96c-f81b-4722-ab7a-e35dfedb081c None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:32:53.566156 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-a1bcb96c-f81b-4722-ab7a-e35dfedb081c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:32:53.673092 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bcb96c-f81b-4722-ab7a-e35dfedb081c None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:32:53.723685 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:32:53.748708 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52ad406-4904-40b6-9342-9a165e7ba1c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:32:53.749317 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52ad406-4904-40b6-9342-9a165e7ba1c6 service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:32:53.749621 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52ad406-4904-40b6-9342-9a165e7ba1c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:32:53.749901 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52ad406-4904-40b6-9342-9a165e7ba1c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:32:53.751491 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-a52ad406-4904-40b6-9342-9a165e7ba1c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:32:53.859113 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52ad406-4904-40b6-9342-9a165e7ba1c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:32:53.860460 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52ad406-4904-40b6-9342-9a165e7ba1c6 service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:32:53.878902 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 717/1455] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:32:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:32:56.143577 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fb36ca-9b5b-400b-a61f-5b4fa4d8a8ee None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:32:56.144137 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fb36ca-9b5b-400b-a61f-5b4fa4d8a8ee None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:32:56.144137 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fb36ca-9b5b-400b-a61f-5b4fa4d8a8ee None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:32:56.145470 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 739/1456] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 07:32:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:32:56.151417 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f76fb49-5ee1-4afe-af3e-835911290903 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:32:56.151809 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f76fb49-5ee1-4afe-af3e-835911290903 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:32:56.152170 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f76fb49-5ee1-4afe-af3e-835911290903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:32:56.218272 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-1f76fb49-5ee1-4afe-af3e-835911290903 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:32:56.219412 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-1f76fb49-5ee1-4afe-af3e-835911290903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:32:56.248636 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:32:56.315834 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 718/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 07:32:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:32:56.602947 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fb36ca-9b5b-400b-a61f-5b4fa4d8a8ee None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:32:56.605267 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-69fb36ca-9b5b-400b-a61f-5b4fa4d8a8ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:32:56.712167 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fb36ca-9b5b-400b-a61f-5b4fa4d8a8ee None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:32:56.741080 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:32:56.786911 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb78602-7ce0-4436-9ed7-811d15bb2fb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:32:56.787516 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb78602-7ce0-4436-9ed7-811d15bb2fb0 service neutron] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:32:56.787798 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb78602-7ce0-4436-9ed7-811d15bb2fb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:32:56.788074 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb78602-7ce0-4436-9ed7-811d15bb2fb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:32:56.789696 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-2cb78602-7ce0-4436-9ed7-811d15bb2fb0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:32:56.902446 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb78602-7ce0-4436-9ed7-811d15bb2fb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:32:56.903808 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb78602-7ce0-4436-9ed7-811d15bb2fb0 service neutron] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:32:56.922686 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 740/1458] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:32:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:32:59.142498 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33172b1a-1d71-4cac-98f7-138eb0218621 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:32:59.143080 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33172b1a-1d71-4cac-98f7-138eb0218621 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:32:59.143080 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33172b1a-1d71-4cac-98f7-138eb0218621 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:32:59.144391 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 719/1459] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 07:32:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:32:59.149530 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe49e38-a837-4f85-9a02-4a18799554d4 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:32:59.149930 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe49e38-a837-4f85-9a02-4a18799554d4 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:32:59.150103 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe49e38-a837-4f85-9a02-4a18799554d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:32:59.214450 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-bfe49e38-a837-4f85-9a02-4a18799554d4 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:32:59.215564 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-bfe49e38-a837-4f85-9a02-4a18799554d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:32:59.223687 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:32:59.310708 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 741/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 07:32:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:32:59.600075 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33172b1a-1d71-4cac-98f7-138eb0218621 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:32:59.601686 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-33172b1a-1d71-4cac-98f7-138eb0218621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:32:59.705753 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33172b1a-1d71-4cac-98f7-138eb0218621 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:32:59.759916 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:32:59.778821 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ac343d-36cd-4b00-b325-6d712bffbe63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:32:59.779420 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ac343d-36cd-4b00-b325-6d712bffbe63 service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:32:59.779696 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ac343d-36cd-4b00-b325-6d712bffbe63 service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:32:59.779967 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ac343d-36cd-4b00-b325-6d712bffbe63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:32:59.781546 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-27ac343d-36cd-4b00-b325-6d712bffbe63 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:32:59.889671 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ac343d-36cd-4b00-b325-6d712bffbe63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:32:59.891081 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ac343d-36cd-4b00-b325-6d712bffbe63 service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:32:59.910003 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 720/1461] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:32:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:33:02.153203 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a3cf1c-2c19-496f-b100-56f1f94c440d None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:33:02.153804 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a3cf1c-2c19-496f-b100-56f1f94c440d None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:33:02.153908 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a3cf1c-2c19-496f-b100-56f1f94c440d None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:33:02.155248 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 742/1462] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 07:33:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:33:02.160649 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a480712-37ad-4ccd-bd12-2aa01ceca9a7 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:33:02.161061 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a480712-37ad-4ccd-bd12-2aa01ceca9a7 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:33:02.161212 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a480712-37ad-4ccd-bd12-2aa01ceca9a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:33:02.226954 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-7a480712-37ad-4ccd-bd12-2aa01ceca9a7 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:33:02.228110 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-7a480712-37ad-4ccd-bd12-2aa01ceca9a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:33:02.267609 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:33:02.328114 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 721/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 07:33:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:33:02.616663 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a3cf1c-2c19-496f-b100-56f1f94c440d None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:33:02.618523 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-38a3cf1c-2c19-496f-b100-56f1f94c440d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:33:02.726625 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a3cf1c-2c19-496f-b100-56f1f94c440d None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:33:02.765818 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:33:02.802286 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb9fb4b-d38b-445e-b9b0-5b87f94ecd16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:33:02.803125 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb9fb4b-d38b-445e-b9b0-5b87f94ecd16 service neutron] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:33:02.803592 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb9fb4b-d38b-445e-b9b0-5b87f94ecd16 service neutron] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:33:02.804136 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb9fb4b-d38b-445e-b9b0-5b87f94ecd16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:33:02.805790 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-7bb9fb4b-d38b-445e-b9b0-5b87f94ecd16 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:33:02.920510 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb9fb4b-d38b-445e-b9b0-5b87f94ecd16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:33:02.921902 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb9fb4b-d38b-445e-b9b0-5b87f94ecd16 service neutron] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:33:02.942686 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 743/1464] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:33:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:33:05.143031 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c0bbd7-db7a-4c8d-8012-96993e9b2303 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:33:05.143598 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c0bbd7-db7a-4c8d-8012-96993e9b2303 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:33:05.143598 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c0bbd7-db7a-4c8d-8012-96993e9b2303 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:33:05.144956 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 722/1465] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 07:33:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:33:05.150261 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3800e94f-dc1d-42ba-81a9-891434032fb9 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:33:05.150688 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3800e94f-dc1d-42ba-81a9-891434032fb9 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:33:05.150831 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3800e94f-dc1d-42ba-81a9-891434032fb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:33:05.217318 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-3800e94f-dc1d-42ba-81a9-891434032fb9 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:33:05.218476 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-3800e94f-dc1d-42ba-81a9-891434032fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:33:05.236818 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:33:05.315944 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 744/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 07:33:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:33:05.597182 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c0bbd7-db7a-4c8d-8012-96993e9b2303 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:33:05.598752 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-22c0bbd7-db7a-4c8d-8012-96993e9b2303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:33:05.705027 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c0bbd7-db7a-4c8d-8012-96993e9b2303 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:33:05.770828 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:33:05.777660 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43de55fb-e1c0-45fc-821e-72cc256ff61d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:33:05.778258 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43de55fb-e1c0-45fc-821e-72cc256ff61d service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:33:05.778537 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43de55fb-e1c0-45fc-821e-72cc256ff61d service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:33:05.778804 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43de55fb-e1c0-45fc-821e-72cc256ff61d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:33:05.780421 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-43de55fb-e1c0-45fc-821e-72cc256ff61d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:33:05.894395 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43de55fb-e1c0-45fc-821e-72cc256ff61d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:33:05.895738 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43de55fb-e1c0-45fc-821e-72cc256ff61d service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:33:05.914090 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 723/1467] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:33:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:33:08.157099 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2a28c5-d847-48de-80a9-6fc5dd218c9e None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:33:08.157738 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2a28c5-d847-48de-80a9-6fc5dd218c9e None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:33:08.157738 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2a28c5-d847-48de-80a9-6fc5dd218c9e None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:33:08.159054 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 745/1468] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 07:33:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:33:08.164431 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cc5c12-5961-4ece-8a96-3a5b44b24bb5 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:33:08.164905 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cc5c12-5961-4ece-8a96-3a5b44b24bb5 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:33:08.165025 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cc5c12-5961-4ece-8a96-3a5b44b24bb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:33:08.232245 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-e8cc5c12-5961-4ece-8a96-3a5b44b24bb5 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:33:08.233367 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-e8cc5c12-5961-4ece-8a96-3a5b44b24bb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:33:08.281301 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:33:08.333878 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 724/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 07:33:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:33:08.625771 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2a28c5-d847-48de-80a9-6fc5dd218c9e None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:33:08.627285 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-6e2a28c5-d847-48de-80a9-6fc5dd218c9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:33:08.730487 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2a28c5-d847-48de-80a9-6fc5dd218c9e None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:33:08.784209 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:33:08.810002 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4de6498-6e6e-415c-a089-f3010c9716e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:33:08.812929 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de6498-6e6e-415c-a089-f3010c9716e1 service neutron] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:33:08.812929 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de6498-6e6e-415c-a089-f3010c9716e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:33:08.812929 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de6498-6e6e-415c-a089-f3010c9716e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:33:08.812929 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-b4de6498-6e6e-415c-a089-f3010c9716e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:33:08.927264 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4de6498-6e6e-415c-a089-f3010c9716e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:33:08.928627 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4de6498-6e6e-415c-a089-f3010c9716e1 service neutron] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:33:08.956675 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 746/1470] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:33:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:33:18.332303 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=22 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:33:18.950220 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=66 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:33:29.466521 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965026ea-9371-47af-9c08-005bb9f3ebb2 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:33:29.467072 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965026ea-9371-47af-9c08-005bb9f3ebb2 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:33:29.467072 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965026ea-9371-47af-9c08-005bb9f3ebb2 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:33:29.468440 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 725/1471] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:33:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:33:29.473720 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4149b60c-d055-4e5a-9d16-91a98e54d59d None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:33:29.474157 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4149b60c-d055-4e5a-9d16-91a98e54d59d None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:33:29.474326 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4149b60c-d055-4e5a-9d16-91a98e54d59d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:33:29.544436 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-4149b60c-d055-4e5a-9d16-91a98e54d59d None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:33:29.545691 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-4149b60c-d055-4e5a-9d16-91a98e54d59d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:33:29.643628 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 747/1472] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:33:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:33:29.953157 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b028de3f-a827-46ea-a67b-6e77d05d194d None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:33:29.953437 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b028de3f-a827-46ea-a67b-6e77d05d194d None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:33:29.953740 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b028de3f-a827-46ea-a67b-6e77d05d194d None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:33:29.955069 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 726/1473] 10.222.0.42 () {54 vars in 962 bytes} [Thu Sep 15 07:33:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:33:29.959939 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc589156-b168-4bd2-b5e7-49d4e984a611 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:33:29.960230 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc589156-b168-4bd2-b5e7-49d4e984a611 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:33:29.960509 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc589156-b168-4bd2-b5e7-49d4e984a611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:33:30.037461 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-fc589156-b168-4bd2-b5e7-49d4e984a611 None None] MFA Rules not processed for user `37438b032a1f46d3bdf7121ad0cffbc3`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:33:30.038854 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-fc589156-b168-4bd2-b5e7-49d4e984a611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:33:30.061731 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:33:30.116762 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 748/1474] 10.222.0.42 () {58 vars in 1059 bytes} [Thu Sep 15 07:33:29 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:33:30.124794 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b028de3f-a827-46ea-a67b-6e77d05d194d None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:33:30.126314 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-b028de3f-a827-46ea-a67b-6e77d05d194d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:33:30.228042 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b028de3f-a827-46ea-a67b-6e77d05d194d None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:33:30.303864 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1375310f-b7ab-4acd-b656-877f043a90cb service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37438b032a1f46d3bdf7121ad0cffbc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:33:30.304471 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1375310f-b7ab-4acd-b656-877f043a90cb service glance] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:33:30.304755 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1375310f-b7ab-4acd-b656-877f043a90cb service glance] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:33:30.305029 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1375310f-b7ab-4acd-b656-877f043a90cb service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:33:30.306639 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-1375310f-b7ab-4acd-b656-877f043a90cb service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:33:30.362163 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:33:30.417495 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1375310f-b7ab-4acd-b656-877f043a90cb service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:33:30.419124 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1375310f-b7ab-4acd-b656-877f043a90cb service glance] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:33:30.437097 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 727/1475] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:33:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:33:32.665873 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c12b9d3-f2f9-4d8d-b1f9-a20d13248eb1 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:33:32.666456 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c12b9d3-f2f9-4d8d-b1f9-a20d13248eb1 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:33:32.666456 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c12b9d3-f2f9-4d8d-b1f9-a20d13248eb1 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:33:32.667781 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 749/1476] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:33:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:33:32.673838 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5cfe6e-de3f-4690-8dcb-f83e341f5403 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:33:32.674233 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5cfe6e-de3f-4690-8dcb-f83e341f5403 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:33:32.674386 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5cfe6e-de3f-4690-8dcb-f83e341f5403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:33:32.739770 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-ae5cfe6e-de3f-4690-8dcb-f83e341f5403 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:33:32.741024 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-ae5cfe6e-de3f-4690-8dcb-f83e341f5403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:33:32.836988 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 728/1477] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:33:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:33:33.121285 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c12b9d3-f2f9-4d8d-b1f9-a20d13248eb1 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:33:33.122853 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-1c12b9d3-f2f9-4d8d-b1f9-a20d13248eb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:33:33.247435 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c12b9d3-f2f9-4d8d-b1f9-a20d13248eb1 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:33:33.298357 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:33:33.343579 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfa3b1c-eadf-4f5c-9744-842d78b95f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:33:33.344191 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfa3b1c-eadf-4f5c-9744-842d78b95f27 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:33:33.344470 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfa3b1c-eadf-4f5c-9744-842d78b95f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:33:33.344741 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfa3b1c-eadf-4f5c-9744-842d78b95f27 admin admin] PATH_INFO: `/v3/projects` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:33:33.345371 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfa3b1c-eadf-4f5c-9744-842d78b95f27 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:33:33.347207 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: WARNING py.warnings [None req-9cfa3b1c-eadf-4f5c-9744-842d78b95f27 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 15 07:33:33.347207 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: warnings.warn(msg) Sep 15 07:33:33.347207 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: Sep 15 07:33:33.348402 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfa3b1c-eadf-4f5c-9744-842d78b95f27 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:33:33.368093 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 750/1478] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 15 07:33:33 2022] GET /identity/v3/projects => generated 2553 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:33:35.552632 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f800b6-b259-4593-97dd-790b29b3dcdf None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:33:35.553215 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f800b6-b259-4593-97dd-790b29b3dcdf None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:33:35.553215 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f800b6-b259-4593-97dd-790b29b3dcdf None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:33:35.554612 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 729/1479] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:33:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:33:35.560158 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e8fcd7-366f-4dbf-8f58-0b977f814e73 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:33:35.560556 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e8fcd7-366f-4dbf-8f58-0b977f814e73 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:33:35.560730 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e8fcd7-366f-4dbf-8f58-0b977f814e73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:33:35.628517 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-99e8fcd7-366f-4dbf-8f58-0b977f814e73 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:33:35.629674 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-99e8fcd7-366f-4dbf-8f58-0b977f814e73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:33:35.727370 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 751/1480] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:33:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:33:36.018782 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f800b6-b259-4593-97dd-790b29b3dcdf None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:33:36.020295 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-46f800b6-b259-4593-97dd-790b29b3dcdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:33:36.030749 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:33:36.169647 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f800b6-b259-4593-97dd-790b29b3dcdf None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:33:36.256619 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:33:36.289669 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25184b9-1ab3-436f-a86f-7c2d2f5548cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a080c7ab9b684abca6828067f2572c3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:33:36.290272 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25184b9-1ab3-436f-a86f-7c2d2f5548cc service nova] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:33:36.290551 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25184b9-1ab3-436f-a86f-7c2d2f5548cc service nova] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:33:36.290821 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25184b9-1ab3-436f-a86f-7c2d2f5548cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:33:36.292410 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-d25184b9-1ab3-436f-a86f-7c2d2f5548cc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:33:36.383001 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25184b9-1ab3-436f-a86f-7c2d2f5548cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:33:36.384569 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25184b9-1ab3-436f-a86f-7c2d2f5548cc service nova] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:33:36.402192 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 730/1481] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:33:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 07:33:38.734139 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b358e523-fa3a-4494-a09c-4c16daec1f85 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:33:38.734712 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b358e523-fa3a-4494-a09c-4c16daec1f85 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:33:38.734712 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b358e523-fa3a-4494-a09c-4c16daec1f85 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:33:38.736015 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 752/1482] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:33:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:33:38.741367 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf13b985-9f16-4e80-a9e5-e0af3ce36d08 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:33:38.741828 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf13b985-9f16-4e80-a9e5-e0af3ce36d08 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:33:38.741943 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf13b985-9f16-4e80-a9e5-e0af3ce36d08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:33:38.808064 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-cf13b985-9f16-4e80-a9e5-e0af3ce36d08 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:33:38.809897 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-cf13b985-9f16-4e80-a9e5-e0af3ce36d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:33:38.854811 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:33:38.910815 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 731/1483] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:33:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:33:39.212782 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b358e523-fa3a-4494-a09c-4c16daec1f85 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:33:39.215125 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-b358e523-fa3a-4494-a09c-4c16daec1f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:33:39.239223 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:33:39.320850 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b358e523-fa3a-4494-a09c-4c16daec1f85 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:33:39.396604 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76af672-399f-4396-a517-faf63df4c7ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:33:39.397211 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76af672-399f-4396-a517-faf63df4c7ab service neutron] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:33:39.397491 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76af672-399f-4396-a517-faf63df4c7ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:33:39.397795 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76af672-399f-4396-a517-faf63df4c7ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:33:39.399372 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-c76af672-399f-4396-a517-faf63df4c7ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:33:39.477234 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:33:39.512909 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76af672-399f-4396-a517-faf63df4c7ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:33:39.514420 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76af672-399f-4396-a517-faf63df4c7ab service neutron] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:33:39.534061 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 753/1484] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:33:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:33:41.798476 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f38a2d5-3e40-4258-afd8-e459c4037810 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:33:41.799072 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f38a2d5-3e40-4258-afd8-e459c4037810 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:33:41.799072 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f38a2d5-3e40-4258-afd8-e459c4037810 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:33:41.800398 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 732/1485] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:33:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:33:41.805631 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ba4306-39ea-4edc-a6a9-aeeaea64df4f None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:33:41.806092 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ba4306-39ea-4edc-a6a9-aeeaea64df4f None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:33:41.806203 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ba4306-39ea-4edc-a6a9-aeeaea64df4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:33:41.872680 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-71ba4306-39ea-4edc-a6a9-aeeaea64df4f None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:33:41.873898 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-71ba4306-39ea-4edc-a6a9-aeeaea64df4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:33:41.969548 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 754/1486] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:33:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:33:42.255797 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f38a2d5-3e40-4258-afd8-e459c4037810 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:33:42.257386 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-6f38a2d5-3e40-4258-afd8-e459c4037810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:33:42.364561 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f38a2d5-3e40-4258-afd8-e459c4037810 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:33:42.435715 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:33:42.437184 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3677bb7-c292-4d52-97c7-e92bd2bc7e2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:33:42.437803 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3677bb7-c292-4d52-97c7-e92bd2bc7e2b service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:33:42.438080 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3677bb7-c292-4d52-97c7-e92bd2bc7e2b service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:33:42.438349 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3677bb7-c292-4d52-97c7-e92bd2bc7e2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:33:42.439909 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-d3677bb7-c292-4d52-97c7-e92bd2bc7e2b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:33:42.549621 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3677bb7-c292-4d52-97c7-e92bd2bc7e2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:33:42.551050 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3677bb7-c292-4d52-97c7-e92bd2bc7e2b service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:33:42.569024 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 733/1487] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:33:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:33:44.831328 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b40288-e6ad-4343-a6e8-7667b6735d7d None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:33:44.847784 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b40288-e6ad-4343-a6e8-7667b6735d7d None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:33:44.847784 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b40288-e6ad-4343-a6e8-7667b6735d7d None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:33:44.847784 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 755/1488] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:33:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:33:44.848109 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ebfbc3-4e6d-48d8-a39a-7badd3eb248b None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:33:44.848109 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ebfbc3-4e6d-48d8-a39a-7badd3eb248b None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:33:44.848109 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ebfbc3-4e6d-48d8-a39a-7badd3eb248b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:33:44.905911 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-56ebfbc3-4e6d-48d8-a39a-7badd3eb248b None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:33:44.907127 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-56ebfbc3-4e6d-48d8-a39a-7badd3eb248b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:33:44.960643 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:33:45.004733 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 734/1489] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:33:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:33:45.294443 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b40288-e6ad-4343-a6e8-7667b6735d7d None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:33:45.295949 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-b3b40288-e6ad-4343-a6e8-7667b6735d7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:33:45.332721 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:33:45.400533 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b40288-e6ad-4343-a6e8-7667b6735d7d None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:33:45.475893 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff97b6b9-80d3-43bf-bd23-b859a4446bc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:33:45.476498 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff97b6b9-80d3-43bf-bd23-b859a4446bc9 service neutron] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:33:45.476781 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff97b6b9-80d3-43bf-bd23-b859a4446bc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:33:45.477055 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff97b6b9-80d3-43bf-bd23-b859a4446bc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:33:45.478681 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-ff97b6b9-80d3-43bf-bd23-b859a4446bc9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:33:45.560652 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:33:45.589740 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff97b6b9-80d3-43bf-bd23-b859a4446bc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:33:45.591717 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff97b6b9-80d3-43bf-bd23-b859a4446bc9 service neutron] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:33:45.726207 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 756/1490] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:33:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:33:48.274256 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b6483b-b6db-4e97-84ab-8290aa2e30cf None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:33:48.274778 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b6483b-b6db-4e97-84ab-8290aa2e30cf None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:33:48.274778 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b6483b-b6db-4e97-84ab-8290aa2e30cf None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:33:48.276083 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 735/1491] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 15 07:33:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:33:48.281835 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aac7608-7e71-4fcd-98e6-0d6e5a926a80 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:33:48.282256 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aac7608-7e71-4fcd-98e6-0d6e5a926a80 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:33:48.282385 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aac7608-7e71-4fcd-98e6-0d6e5a926a80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:33:48.355002 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-4aac7608-7e71-4fcd-98e6-0d6e5a926a80 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:33:48.356204 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-4aac7608-7e71-4fcd-98e6-0d6e5a926a80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:33:48.452633 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 757/1492] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 15 07:33:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:33:48.740635 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b6483b-b6db-4e97-84ab-8290aa2e30cf None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:33:48.742236 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-c3b6483b-b6db-4e97-84ab-8290aa2e30cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:33:48.846938 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b6483b-b6db-4e97-84ab-8290aa2e30cf None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:33:48.921469 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f965133-901f-4f71-8b4b-b0a26854d5b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:33:48.922098 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f965133-901f-4f71-8b4b-b0a26854d5b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:33:48.922433 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f965133-901f-4f71-8b4b-b0a26854d5b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:33:48.922707 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f965133-901f-4f71-8b4b-b0a26854d5b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:33:48.924295 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-3f965133-901f-4f71-8b4b-b0a26854d5b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:33:48.932816 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:33:49.039507 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f965133-901f-4f71-8b4b-b0a26854d5b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:33:49.040846 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f965133-901f-4f71-8b4b-b0a26854d5b6 service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:33:49.060475 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 736/1493] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:33:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:33:58.449925 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=82 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:33:59.057437 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=50 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:35:11.033420 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34afced3-4ccb-408b-adeb-2fc018709284 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:35:11.034016 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34afced3-4ccb-408b-adeb-2fc018709284 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:35:11.034016 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34afced3-4ccb-408b-adeb-2fc018709284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:35:11.110237 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-34afced3-4ccb-408b-adeb-2fc018709284 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:35:11.111444 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-34afced3-4ccb-408b-adeb-2fc018709284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:35:11.206775 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 758/1494] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:35:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:35:11.215112 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b1fbf1-a280-44f4-960f-a1a5b77c3ce4 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:35:11.215594 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b1fbf1-a280-44f4-960f-a1a5b77c3ce4 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:35:11.215695 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b1fbf1-a280-44f4-960f-a1a5b77c3ce4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:35:11.290132 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-32b1fbf1-a280-44f4-960f-a1a5b77c3ce4 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:35:11.291331 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-32b1fbf1-a280-44f4-960f-a1a5b77c3ce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:35:11.386703 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 737/1495] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:35:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 07:35:11.391402 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34afced3-4ccb-408b-adeb-2fc018709284 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:35:11.392604 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-34afced3-4ccb-408b-adeb-2fc018709284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:35:11.404565 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34afced3-4ccb-408b-adeb-2fc018709284 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:35:11.466997 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:35:11.499658 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ecc0f9-d40d-45a1-a41b-ddbdddc4ef01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:35:11.500381 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ecc0f9-d40d-45a1-a41b-ddbdddc4ef01 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:35:11.500661 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ecc0f9-d40d-45a1-a41b-ddbdddc4ef01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:35:11.500933 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ecc0f9-d40d-45a1-a41b-ddbdddc4ef01 admin admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:35:11.501568 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ecc0f9-d40d-45a1-a41b-ddbdddc4ef01 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:35:11.501900 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ecc0f9-d40d-45a1-a41b-ddbdddc4ef01 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:35:11.502926 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: WARNING py.warnings [None req-e4ecc0f9-d40d-45a1-a41b-ddbdddc4ef01 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 15 07:35:11.502926 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: warnings.warn(msg) Sep 15 07:35:11.502926 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: Sep 15 07:35:11.504622 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ecc0f9-d40d-45a1-a41b-ddbdddc4ef01 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:35:11.517790 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 759/1496] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 15 07:35:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:35:11.526239 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b1fbf1-a280-44f4-960f-a1a5b77c3ce4 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:35:11.535215 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b1fbf1-a280-44f4-960f-a1a5b77c3ce4 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:35:11.595306 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:35:11.629125 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf44c8d7-5797-416a-afa6-61572bbb3a21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:35:11.629969 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf44c8d7-5797-416a-afa6-61572bbb3a21 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:35:11.630269 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf44c8d7-5797-416a-afa6-61572bbb3a21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:35:11.630543 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf44c8d7-5797-416a-afa6-61572bbb3a21 admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:35:11.631176 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf44c8d7-5797-416a-afa6-61572bbb3a21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:35:11.632924 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: WARNING py.warnings [None req-bf44c8d7-5797-416a-afa6-61572bbb3a21 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 15 07:35:11.632924 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: warnings.warn(msg) Sep 15 07:35:11.632924 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: Sep 15 07:35:11.634135 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf44c8d7-5797-416a-afa6-61572bbb3a21 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:35:11.744994 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 738/1497] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:35:11 2022] POST /identity/v3/projects => generated 377 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:35:11.749130 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ecc0f9-d40d-45a1-a41b-ddbdddc4ef01 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:35:11.750638 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-e4ecc0f9-d40d-45a1-a41b-ddbdddc4ef01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:35:11.785014 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ecc0f9-d40d-45a1-a41b-ddbdddc4ef01 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:35:11.881446 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70450ef-4bb0-4e2f-af97-c155db05664a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:35:11.882077 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70450ef-4bb0-4e2f-af97-c155db05664a admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:35:11.882376 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70450ef-4bb0-4e2f-af97-c155db05664a admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:35:11.882641 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70450ef-4bb0-4e2f-af97-c155db05664a admin admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:35:11.883270 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70450ef-4bb0-4e2f-af97-c155db05664a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:35:11.884296 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: WARNING py.warnings [None req-e70450ef-4bb0-4e2f-af97-c155db05664a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 15 07:35:11.884296 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: warnings.warn(msg) Sep 15 07:35:11.884296 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: Sep 15 07:35:11.885485 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70450ef-4bb0-4e2f-af97-c155db05664a admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:35:12.017755 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 760/1498] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:35:11 2022] POST /identity/v3/users => generated 289 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:35:12.021821 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf44c8d7-5797-416a-afa6-61572bbb3a21 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:35:12.023325 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-bf44c8d7-5797-416a-afa6-61572bbb3a21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:35:12.085039 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf44c8d7-5797-416a-afa6-61572bbb3a21 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:35:12.167318 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:35:12.177284 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106967b5-bf9c-4631-abfd-8be513a6d623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:35:12.180503 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106967b5-bf9c-4631-abfd-8be513a6d623 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:35:12.180503 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106967b5-bf9c-4631-abfd-8be513a6d623 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:35:12.180503 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106967b5-bf9c-4631-abfd-8be513a6d623 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:35:12.180503 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106967b5-bf9c-4631-abfd-8be513a6d623 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:35:12.180503 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: WARNING py.warnings [None req-106967b5-bf9c-4631-abfd-8be513a6d623 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 15 07:35:12.180503 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: warnings.warn(msg) Sep 15 07:35:12.180503 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: Sep 15 07:35:12.181122 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106967b5-bf9c-4631-abfd-8be513a6d623 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:35:12.193595 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 739/1499] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:35:12 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:35:12.197565 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70450ef-4bb0-4e2f-af97-c155db05664a admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:35:12.199103 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-e70450ef-4bb0-4e2f-af97-c155db05664a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:35:12.211240 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70450ef-4bb0-4e2f-af97-c155db05664a admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:35:12.236523 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:35:12.322951 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2500a60-a56c-4950-9da6-624e2dc88a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:35:12.323564 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2500a60-a56c-4950-9da6-624e2dc88a01 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:35:12.323840 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2500a60-a56c-4950-9da6-624e2dc88a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:35:12.324114 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2500a60-a56c-4950-9da6-624e2dc88a01 admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:35:12.324845 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2500a60-a56c-4950-9da6-624e2dc88a01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:35:12.325878 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: WARNING py.warnings [None req-f2500a60-a56c-4950-9da6-624e2dc88a01 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 15 07:35:12.325878 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: warnings.warn(msg) Sep 15 07:35:12.325878 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: Sep 15 07:35:12.326956 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2500a60-a56c-4950-9da6-624e2dc88a01 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:35:12.345223 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 761/1500] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:35:12 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:35:12.350772 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106967b5-bf9c-4631-abfd-8be513a6d623 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:35:12.352295 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-106967b5-bf9c-4631-abfd-8be513a6d623 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:35:12.386699 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106967b5-bf9c-4631-abfd-8be513a6d623 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:35:12.482911 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d50750d-2e94-45c4-819c-c8a448875ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:35:12.483572 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d50750d-2e94-45c4-819c-c8a448875ff4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:35:12.483856 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d50750d-2e94-45c4-819c-c8a448875ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:35:12.484134 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d50750d-2e94-45c4-819c-c8a448875ff4 admin admin] PATH_INFO: `/v3/projects/4bb242fff5a24fb081c3247d2b31822a/users/3644029ff32e454cb398784093f2276b/roles/053e12af84e943e699460a9d76c2072b` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:35:12.507019 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d50750d-2e94-45c4-819c-c8a448875ff4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bb242fff5a24fb081c3247d2b31822a, user_id=3644029ff32e454cb398784093f2276b, role_id=053e12af84e943e699460a9d76c2072b)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:35:12.508039 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: WARNING py.warnings [None req-1d50750d-2e94-45c4-819c-c8a448875ff4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 15 07:35:12.508039 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: warnings.warn(msg) Sep 15 07:35:12.508039 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: Sep 15 07:35:12.510025 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d50750d-2e94-45c4-819c-c8a448875ff4 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:35:12.623372 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 740/1501] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:35:12 2022] PUT /identity/v3/projects/4bb242fff5a24fb081c3247d2b31822a/users/3644029ff32e454cb398784093f2276b/roles/053e12af84e943e699460a9d76c2072b => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:35:12.636525 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59851ad3-aea0-4fa1-a9b3-9277678e637f None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:35:12.636891 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59851ad3-aea0-4fa1-a9b3-9277678e637f None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:35:12.637243 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59851ad3-aea0-4fa1-a9b3-9277678e637f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:35:12.698290 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-59851ad3-aea0-4fa1-a9b3-9277678e637f None None] MFA Rules not processed for user `3644029ff32e454cb398784093f2276b`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:35:12.699546 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-59851ad3-aea0-4fa1-a9b3-9277678e637f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:35:12.777012 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 762/1502] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:35:12 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:35:12.790647 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2a3cee-c325-4c95-b09d-6e8964146fe2 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:35:12.790941 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2a3cee-c325-4c95-b09d-6e8964146fe2 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:35:12.791248 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2a3cee-c325-4c95-b09d-6e8964146fe2 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:35:12.792596 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 741/1503] 10.222.0.42 () {54 vars in 964 bytes} [Thu Sep 15 07:35:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:35:12.797554 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d1ae60-7eff-4f30-9416-d72236f9f5a0 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:35:12.797876 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d1ae60-7eff-4f30-9416-d72236f9f5a0 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:35:12.798153 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d1ae60-7eff-4f30-9416-d72236f9f5a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:35:12.831046 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:35:12.877004 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-26d1ae60-7eff-4f30-9416-d72236f9f5a0 None None] MFA Rules not processed for user `3ecf6642b7b84d3c9756bb41ab3349d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:35:12.878568 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-26d1ae60-7eff-4f30-9416-d72236f9f5a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:35:12.954052 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 763/1504] 10.222.0.42 () {58 vars in 1061 bytes} [Thu Sep 15 07:35:12 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:35:12.962512 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2a3cee-c325-4c95-b09d-6e8964146fe2 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:35:12.964027 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-df2a3cee-c325-4c95-b09d-6e8964146fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:35:13.014042 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:35:13.068384 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2a3cee-c325-4c95-b09d-6e8964146fe2 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:35:13.141271 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec26d502-866f-491a-b318-c252dbd1801a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecf6642b7b84d3c9756bb41ab3349d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:35:13.141833 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec26d502-866f-491a-b318-c252dbd1801a service cinder] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:35:13.142111 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec26d502-866f-491a-b318-c252dbd1801a service cinder] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:35:13.142375 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec26d502-866f-491a-b318-c252dbd1801a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:35:13.144521 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-ec26d502-866f-491a-b318-c252dbd1801a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:35:13.244409 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec26d502-866f-491a-b318-c252dbd1801a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:35:13.245886 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec26d502-866f-491a-b318-c252dbd1801a service cinder] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:35:13.249337 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:35:13.263947 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 742/1505] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:35:12 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:35:13.309520 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7751bae6-47d0-44ea-b9d0-aa7176b9ac15 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:35:13.310039 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7751bae6-47d0-44ea-b9d0-aa7176b9ac15 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:35:13.310150 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7751bae6-47d0-44ea-b9d0-aa7176b9ac15 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:35:13.311394 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 764/1506] 10.222.0.42 () {52 vars in 830 bytes} [Thu Sep 15 07:35:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:35:13.314980 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec26d502-866f-491a-b318-c252dbd1801a service cinder] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:35:13.316726 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-ec26d502-866f-491a-b318-c252dbd1801a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:35:13.416711 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec26d502-866f-491a-b318-c252dbd1801a service cinder] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:35:13.517717 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573b4d75-1da5-48c3-86a5-b26e8fd566f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:35:13.518349 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573b4d75-1da5-48c3-86a5-b26e8fd566f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:35:13.518630 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573b4d75-1da5-48c3-86a5-b26e8fd566f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:35:13.520066 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573b4d75-1da5-48c3-86a5-b26e8fd566f4 admin admin] PATH_INFO: `/v3/users/3644029ff32e454cb398784093f2276b` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:35:13.532986 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573b4d75-1da5-48c3-86a5-b26e8fd566f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3644029ff32e454cb398784093f2276b)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:35:13.534023 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: WARNING py.warnings [None req-573b4d75-1da5-48c3-86a5-b26e8fd566f4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 15 07:35:13.534023 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: warnings.warn(msg) Sep 15 07:35:13.534023 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: Sep 15 07:35:13.535808 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573b4d75-1da5-48c3-86a5-b26e8fd566f4 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:35:13.545572 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:35:13.743184 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-573b4d75-1da5-48c3-86a5-b26e8fd566f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3644029ff32e454cb398784093f2276b'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:35:13.743645 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-573b4d75-1da5-48c3-86a5-b26e8fd566f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3644029ff32e454cb398784093f2276b'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:35:13.816668 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-573b4d75-1da5-48c3-86a5-b26e8fd566f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3644029ff32e454cb398784093f2276b'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:35:13.817100 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-573b4d75-1da5-48c3-86a5-b26e8fd566f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3644029ff32e454cb398784093f2276b'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:35:13.833297 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-573b4d75-1da5-48c3-86a5-b26e8fd566f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3644029ff32e454cb398784093f2276b'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:35:13.861461 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 743/1507] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:35:13 2022] DELETE /identity/v3/users/3644029ff32e454cb398784093f2276b => generated 0 bytes in 547 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:35:13.866443 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fa8720-3f4b-4cbb-bda8-5eb73064676d None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:35:13.866724 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fa8720-3f4b-4cbb-bda8-5eb73064676d None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:35:13.866995 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fa8720-3f4b-4cbb-bda8-5eb73064676d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:35:13.922921 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-a8fa8720-3f4b-4cbb-bda8-5eb73064676d None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:35:13.924107 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-a8fa8720-3f4b-4cbb-bda8-5eb73064676d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:35:14.002520 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:35:14.015157 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 765/1508] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:35:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:35:14.024888 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573b4d75-1da5-48c3-86a5-b26e8fd566f4 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:35:14.026134 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-573b4d75-1da5-48c3-86a5-b26e8fd566f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:35:14.124992 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573b4d75-1da5-48c3-86a5-b26e8fd566f4 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:35:14.179528 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:35:14.199058 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1606d97-384e-4220-a59d-58936a9075f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:35:14.199667 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1606d97-384e-4220-a59d-58936a9075f4 service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:35:14.199950 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1606d97-384e-4220-a59d-58936a9075f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:35:14.200231 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1606d97-384e-4220-a59d-58936a9075f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:35:14.201843 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-d1606d97-384e-4220-a59d-58936a9075f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:35:14.312808 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1606d97-384e-4220-a59d-58936a9075f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:35:14.314272 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1606d97-384e-4220-a59d-58936a9075f4 service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:35:14.333226 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 744/1509] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:35:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:35:14.813159 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fa8720-3f4b-4cbb-bda8-5eb73064676d None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:35:14.814434 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-a8fa8720-3f4b-4cbb-bda8-5eb73064676d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:35:14.832208 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fa8720-3f4b-4cbb-bda8-5eb73064676d None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:35:14.926710 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cd1af5-7fa1-4423-b32d-f50f8edfa1d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:35:14.927344 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cd1af5-7fa1-4423-b32d-f50f8edfa1d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:35:14.927647 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cd1af5-7fa1-4423-b32d-f50f8edfa1d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:35:14.927918 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cd1af5-7fa1-4423-b32d-f50f8edfa1d4 admin admin] PATH_INFO: `/v3/projects/4bb242fff5a24fb081c3247d2b31822a` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:35:14.937777 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cd1af5-7fa1-4423-b32d-f50f8edfa1d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4bb242fff5a24fb081c3247d2b31822a)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:35:14.938813 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: WARNING py.warnings [None req-01cd1af5-7fa1-4423-b32d-f50f8edfa1d4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 15 07:35:14.938813 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: warnings.warn(msg) Sep 15 07:35:14.938813 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: Sep 15 07:35:14.940004 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cd1af5-7fa1-4423-b32d-f50f8edfa1d4 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:35:15.045533 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-01cd1af5-7fa1-4423-b32d-f50f8edfa1d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4bb242fff5a24fb081c3247d2b31822a', 'request_id': 'req-01cd1af5-7fa1-4423-b32d-f50f8edfa1d4'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:35:15.058968 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-01cd1af5-7fa1-4423-b32d-f50f8edfa1d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4bb242fff5a24fb081c3247d2b31822a', 'request_id': 'req-01cd1af5-7fa1-4423-b32d-f50f8edfa1d4'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:35:15.136962 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-01cd1af5-7fa1-4423-b32d-f50f8edfa1d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4bb242fff5a24fb081c3247d2b31822a', 'request_id': 'req-01cd1af5-7fa1-4423-b32d-f50f8edfa1d4'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:35:15.160573 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-01cd1af5-7fa1-4423-b32d-f50f8edfa1d4 admin admin] The token cache is being invalidate because project 4bb242fff5a24fb081c3247d2b31822a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:35:15.160865 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-01cd1af5-7fa1-4423-b32d-f50f8edfa1d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:35:15.161518 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-01cd1af5-7fa1-4423-b32d-f50f8edfa1d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:35:15.162843 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 766/1510] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:35:14 2022] DELETE /identity/v3/projects/4bb242fff5a24fb081c3247d2b31822a => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:35:24.330799 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=62 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:35:25.158525 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:36:36.785079 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90856977-76b2-41b8-ba48-a9f96b1fa2f1 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:36:36.785733 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90856977-76b2-41b8-ba48-a9f96b1fa2f1 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:36:36.785733 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90856977-76b2-41b8-ba48-a9f96b1fa2f1 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:36:36.787058 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 745/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 15 07:36:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:36:36.792215 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab3d507-7689-40e0-9492-97feaa63bca8 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:36:36.792647 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab3d507-7689-40e0-9492-97feaa63bca8 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:36:36.792752 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab3d507-7689-40e0-9492-97feaa63bca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:36:36.874152 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-aab3d507-7689-40e0-9492-97feaa63bca8 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:36:36.875354 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-aab3d507-7689-40e0-9492-97feaa63bca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:36:36.980080 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 767/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 15 07:36:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:36:37.137253 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90856977-76b2-41b8-ba48-a9f96b1fa2f1 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:36:37.139380 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-90856977-76b2-41b8-ba48-a9f96b1fa2f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:36:37.258664 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90856977-76b2-41b8-ba48-a9f96b1fa2f1 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:36:37.264472 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3913e4-822f-4498-a73e-80e6a9a14eaf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a080c7ab9b684abca6828067f2572c3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:36:37.265082 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3913e4-822f-4498-a73e-80e6a9a14eaf service nova] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:36:37.265366 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3913e4-822f-4498-a73e-80e6a9a14eaf service nova] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:36:37.265680 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3913e4-822f-4498-a73e-80e6a9a14eaf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:36:37.273703 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3913e4-822f-4498-a73e-80e6a9a14eaf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:36:37.275278 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3913e4-822f-4498-a73e-80e6a9a14eaf service nova] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:36:37.285016 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 746/1513] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:36:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:36:38.418622 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca353110-dd19-42b4-adbd-0d534da75b13 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:36:38.419161 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca353110-dd19-42b4-adbd-0d534da75b13 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:36:38.419161 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca353110-dd19-42b4-adbd-0d534da75b13 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:36:38.420610 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 768/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 15 07:36:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 07:36:38.426939 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ab65c1-7f27-44e3-9471-d53802bf0df5 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:36:38.427385 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ab65c1-7f27-44e3-9471-d53802bf0df5 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:36:38.427489 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ab65c1-7f27-44e3-9471-d53802bf0df5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:36:38.448664 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-05ab65c1-7f27-44e3-9471-d53802bf0df5 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:36:38.449903 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-05ab65c1-7f27-44e3-9471-d53802bf0df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:36:38.459165 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 747/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 15 07:36:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:36:38.621561 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca353110-dd19-42b4-adbd-0d534da75b13 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:36:38.636691 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca353110-dd19-42b4-adbd-0d534da75b13 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:36:38.642108 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b66bd7-ace9-41c2-b094-1f81fda062cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a080c7ab9b684abca6828067f2572c3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:36:38.642712 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b66bd7-ace9-41c2-b094-1f81fda062cc service nova] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:36:38.642982 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b66bd7-ace9-41c2-b094-1f81fda062cc service nova] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:36:38.643254 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b66bd7-ace9-41c2-b094-1f81fda062cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:36:38.651474 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b66bd7-ace9-41c2-b094-1f81fda062cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:36:38.653044 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b66bd7-ace9-41c2-b094-1f81fda062cc service nova] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:36:38.662213 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 769/1516] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:36:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:36:38.958497 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9383c560-c95e-45c5-911d-fabceef02307 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:36:38.958788 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9383c560-c95e-45c5-911d-fabceef02307 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:36:38.959068 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9383c560-c95e-45c5-911d-fabceef02307 None None] PATH_INFO: `/v3` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:36:38.960359 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 748/1517] 10.222.0.26 () {54 vars in 925 bytes} [Thu Sep 15 07:36:38 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:36:38.966355 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d48e0b-a1c5-4893-8743-6bb1a3342f49 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:36:38.966653 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d48e0b-a1c5-4893-8743-6bb1a3342f49 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:36:38.966934 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d48e0b-a1c5-4893-8743-6bb1a3342f49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:36:39.027412 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-85d48e0b-a1c5-4893-8743-6bb1a3342f49 None None] MFA Rules not processed for user `dd87846b9a134248b28e0521b0a880c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:36:39.028609 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-85d48e0b-a1c5-4893-8743-6bb1a3342f49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:36:39.046097 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:36:39.101484 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 770/1518] 10.222.0.26 () {58 vars in 1015 bytes} [Thu Sep 15 07:36:38 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:36:42.798337 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7864b5c-b164-487c-b38d-8b00a9f9d4fd None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:36:42.798796 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7864b5c-b164-487c-b38d-8b00a9f9d4fd None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:36:42.798891 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7864b5c-b164-487c-b38d-8b00a9f9d4fd None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:36:42.800201 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 749/1519] 10.222.0.42 () {54 vars in 961 bytes} [Thu Sep 15 07:36:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:36:42.805285 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf147a6-a621-4f13-be26-0e1ec14eed8f None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:36:42.805746 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf147a6-a621-4f13-be26-0e1ec14eed8f None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:36:42.805898 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf147a6-a621-4f13-be26-0e1ec14eed8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:36:42.827703 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-9cf147a6-a621-4f13-be26-0e1ec14eed8f None None] MFA Rules not processed for user `dd87846b9a134248b28e0521b0a880c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:36:42.828895 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-9cf147a6-a621-4f13-be26-0e1ec14eed8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:36:42.837729 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 771/1520] 10.222.0.42 () {58 vars in 1058 bytes} [Thu Sep 15 07:36:42 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:36:42.849072 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7864b5c-b164-487c-b38d-8b00a9f9d4fd None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:36:42.864125 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7864b5c-b164-487c-b38d-8b00a9f9d4fd None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:36:42.869166 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbcd863-e758-447b-9f97-8fb3d5a23bc7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd87846b9a134248b28e0521b0a880c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:36:42.869864 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbcd863-e758-447b-9f97-8fb3d5a23bc7 service placement] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:36:42.870343 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbcd863-e758-447b-9f97-8fb3d5a23bc7 service placement] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:36:42.870710 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbcd863-e758-447b-9f97-8fb3d5a23bc7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:36:42.878877 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbcd863-e758-447b-9f97-8fb3d5a23bc7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:36:42.880436 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbcd863-e758-447b-9f97-8fb3d5a23bc7 service placement] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:36:42.890172 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 750/1521] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:36:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:36:50.139764 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c0cbfe-8fd4-4f47-b8ec-335274d199f2 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:36:50.140360 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c0cbfe-8fd4-4f47-b8ec-335274d199f2 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:36:50.140360 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c0cbfe-8fd4-4f47-b8ec-335274d199f2 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:36:50.141648 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 772/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 15 07:36:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:36:50.147632 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7394d51-b96b-4350-93fb-71f44bc4f8bf None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:36:50.148069 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7394d51-b96b-4350-93fb-71f44bc4f8bf None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:36:50.148175 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7394d51-b96b-4350-93fb-71f44bc4f8bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:36:50.169238 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-a7394d51-b96b-4350-93fb-71f44bc4f8bf None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:36:50.170475 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-a7394d51-b96b-4350-93fb-71f44bc4f8bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:36:50.178278 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 751/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 15 07:36:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:36:50.340140 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c0cbfe-8fd4-4f47-b8ec-335274d199f2 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:36:50.354757 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c0cbfe-8fd4-4f47-b8ec-335274d199f2 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:36:50.360340 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73cfdb2-7fe5-4186-9a97-9b61a1799fad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a080c7ab9b684abca6828067f2572c3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:36:50.360927 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73cfdb2-7fe5-4186-9a97-9b61a1799fad service nova] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:36:50.361205 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73cfdb2-7fe5-4186-9a97-9b61a1799fad service nova] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:36:50.361471 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73cfdb2-7fe5-4186-9a97-9b61a1799fad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:36:50.369423 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73cfdb2-7fe5-4186-9a97-9b61a1799fad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:36:50.371007 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73cfdb2-7fe5-4186-9a97-9b61a1799fad service nova] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:36:50.381376 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 773/1524] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:36:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:36:51.475946 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a16c317-058b-4969-9776-3fe7ccd73295 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:36:51.476367 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a16c317-058b-4969-9776-3fe7ccd73295 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:36:51.476566 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a16c317-058b-4969-9776-3fe7ccd73295 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:36:51.477961 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 752/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 15 07:36:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:36:51.484341 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9323ae-9171-44bb-90ec-55e6606a1d9d None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:36:51.484788 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9323ae-9171-44bb-90ec-55e6606a1d9d None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:36:51.484893 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9323ae-9171-44bb-90ec-55e6606a1d9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:36:51.503964 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-9a9323ae-9171-44bb-90ec-55e6606a1d9d None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:36:51.505154 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-9a9323ae-9171-44bb-90ec-55e6606a1d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:36:51.513260 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 774/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 15 07:36:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 07:36:51.676125 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a16c317-058b-4969-9776-3fe7ccd73295 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:36:51.693708 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a16c317-058b-4969-9776-3fe7ccd73295 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:36:51.699182 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eff7232-a732-43ec-8165-673060dfe315 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a080c7ab9b684abca6828067f2572c3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:36:51.699786 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eff7232-a732-43ec-8165-673060dfe315 service nova] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:36:51.700070 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eff7232-a732-43ec-8165-673060dfe315 service nova] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:36:51.700349 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eff7232-a732-43ec-8165-673060dfe315 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:36:51.709414 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eff7232-a732-43ec-8165-673060dfe315 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:36:51.711012 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eff7232-a732-43ec-8165-673060dfe315 service nova] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:36:51.721002 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 753/1527] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:36:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:36:55.809403 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88a9f31-d020-451c-a25c-bc7b3ed09f70 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:36:55.810035 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88a9f31-d020-451c-a25c-bc7b3ed09f70 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:36:55.810035 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88a9f31-d020-451c-a25c-bc7b3ed09f70 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:36:55.811797 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 775/1528] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 07:36:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:36:55.818069 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6841ed-4d4a-4863-a0b0-1127b537a5ca None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:36:55.818568 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6841ed-4d4a-4863-a0b0-1127b537a5ca None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:36:55.818798 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6841ed-4d4a-4863-a0b0-1127b537a5ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:36:55.837893 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-ee6841ed-4d4a-4863-a0b0-1127b537a5ca None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:36:55.839076 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-ee6841ed-4d4a-4863-a0b0-1127b537a5ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:36:55.847443 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 754/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 07:36:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:36:56.143928 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88a9f31-d020-451c-a25c-bc7b3ed09f70 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:36:56.145690 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-e88a9f31-d020-451c-a25c-bc7b3ed09f70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:36:56.223777 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88a9f31-d020-451c-a25c-bc7b3ed09f70 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:36:56.227355 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba7cf8d-e050-4d19-87f7-4e8cc98869d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:36:56.227958 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba7cf8d-e050-4d19-87f7-4e8cc98869d1 service neutron] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:36:56.228240 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba7cf8d-e050-4d19-87f7-4e8cc98869d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:36:56.228519 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba7cf8d-e050-4d19-87f7-4e8cc98869d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:36:56.237539 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba7cf8d-e050-4d19-87f7-4e8cc98869d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:36:56.239004 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba7cf8d-e050-4d19-87f7-4e8cc98869d1 service neutron] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:36:56.248900 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 776/1530] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:36:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:36:58.392105 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ce53f6-c8c1-4175-bf3f-8b27faeec0a7 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:36:58.392989 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ce53f6-c8c1-4175-bf3f-8b27faeec0a7 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:36:58.392989 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ce53f6-c8c1-4175-bf3f-8b27faeec0a7 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:36:58.394127 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 755/1531] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 07:36:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:36:58.399370 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a44ba2-b335-4b29-987c-899d3d037bf5 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:36:58.405002 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a44ba2-b335-4b29-987c-899d3d037bf5 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:36:58.405002 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a44ba2-b335-4b29-987c-899d3d037bf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:36:58.419593 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-85a44ba2-b335-4b29-987c-899d3d037bf5 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:36:58.420852 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-85a44ba2-b335-4b29-987c-899d3d037bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:36:58.428625 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 777/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 07:36:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:36:58.716556 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ce53f6-c8c1-4175-bf3f-8b27faeec0a7 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:36:58.718439 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-07ce53f6-c8c1-4175-bf3f-8b27faeec0a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:36:58.732112 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ce53f6-c8c1-4175-bf3f-8b27faeec0a7 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:36:58.736026 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c53c310-1253-4f5c-90aa-3a52d33742de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:36:58.736627 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c53c310-1253-4f5c-90aa-3a52d33742de service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:36:58.736910 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c53c310-1253-4f5c-90aa-3a52d33742de service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:36:58.737186 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c53c310-1253-4f5c-90aa-3a52d33742de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:36:58.746043 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c53c310-1253-4f5c-90aa-3a52d33742de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:36:58.747461 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c53c310-1253-4f5c-90aa-3a52d33742de service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:36:58.757330 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 756/1533] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:36:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:37:02.401311 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dbf97a-602d-4b3f-a050-60b74078b055 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:37:02.401879 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dbf97a-602d-4b3f-a050-60b74078b055 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:37:02.401879 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dbf97a-602d-4b3f-a050-60b74078b055 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:37:02.403224 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 778/1534] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 07:37:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:37:02.408501 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af649def-89dd-4c90-b9f7-e253714f4bcf None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:37:02.408957 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af649def-89dd-4c90-b9f7-e253714f4bcf None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:37:02.409067 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af649def-89dd-4c90-b9f7-e253714f4bcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:37:02.428244 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-af649def-89dd-4c90-b9f7-e253714f4bcf None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:37:02.429424 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-af649def-89dd-4c90-b9f7-e253714f4bcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:37:02.437921 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 757/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 07:37:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:37:02.741785 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dbf97a-602d-4b3f-a050-60b74078b055 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:37:02.743469 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-25dbf97a-602d-4b3f-a050-60b74078b055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:37:02.807273 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:37:02.820788 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dbf97a-602d-4b3f-a050-60b74078b055 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:37:02.824371 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0335ee9-1018-4b8f-b91a-3c09aabb547e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecf6642b7b84d3c9756bb41ab3349d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:37:02.824971 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0335ee9-1018-4b8f-b91a-3c09aabb547e service cinder] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:37:02.825249 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0335ee9-1018-4b8f-b91a-3c09aabb547e service cinder] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:37:02.825526 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0335ee9-1018-4b8f-b91a-3c09aabb547e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:37:02.834510 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0335ee9-1018-4b8f-b91a-3c09aabb547e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:37:02.835927 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0335ee9-1018-4b8f-b91a-3c09aabb547e service cinder] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:37:02.846217 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 779/1536] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:37:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:37:05.010790 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba44992a-bf19-4ea4-a505-b23a270abdcf None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:37:05.011340 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba44992a-bf19-4ea4-a505-b23a270abdcf None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:37:05.011340 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba44992a-bf19-4ea4-a505-b23a270abdcf None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:37:05.012942 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 758/1537] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 07:37:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:37:05.018853 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59d04d1-3e77-453e-9a33-aaf10cb7f916 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:37:05.019300 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59d04d1-3e77-453e-9a33-aaf10cb7f916 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:37:05.019403 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59d04d1-3e77-453e-9a33-aaf10cb7f916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:37:05.039291 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-c59d04d1-3e77-453e-9a33-aaf10cb7f916 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:37:05.040476 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-c59d04d1-3e77-453e-9a33-aaf10cb7f916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:37:05.048395 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 780/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 07:37:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:37:05.350554 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba44992a-bf19-4ea4-a505-b23a270abdcf None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:37:05.352187 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-ba44992a-bf19-4ea4-a505-b23a270abdcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:37:05.359816 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=84 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:37:05.365838 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba44992a-bf19-4ea4-a505-b23a270abdcf None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:37:05.369324 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6154b001-90fb-4286-8974-289b228fd72f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecf6642b7b84d3c9756bb41ab3349d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:37:05.369969 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6154b001-90fb-4286-8974-289b228fd72f service cinder] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:37:05.386033 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6154b001-90fb-4286-8974-289b228fd72f service cinder] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:37:05.386033 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6154b001-90fb-4286-8974-289b228fd72f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:37:05.386033 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6154b001-90fb-4286-8974-289b228fd72f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:37:05.386033 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6154b001-90fb-4286-8974-289b228fd72f service cinder] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:37:05.391262 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 759/1539] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:37:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:37:15.034419 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=11 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:37:15.405952 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=5 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:38:28.060636 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add2dd17-114a-404d-8972-54eefac75015 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:28.062066 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add2dd17-114a-404d-8972-54eefac75015 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:28.062066 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add2dd17-114a-404d-8972-54eefac75015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:28.098874 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-add2dd17-114a-404d-8972-54eefac75015 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:38:28.100160 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-add2dd17-114a-404d-8972-54eefac75015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:28.111386 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 781/1540] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:38:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 07:38:28.120805 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb948db-167b-4d8f-96af-ed7cf78f4634 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:28.122005 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb948db-167b-4d8f-96af-ed7cf78f4634 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:28.129686 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb948db-167b-4d8f-96af-ed7cf78f4634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:28.150954 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-aeb948db-167b-4d8f-96af-ed7cf78f4634 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:38:28.152166 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-aeb948db-167b-4d8f-96af-ed7cf78f4634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:28.170192 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 760/1541] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:38:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:38:28.174851 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add2dd17-114a-404d-8972-54eefac75015 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:28.190901 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add2dd17-114a-404d-8972-54eefac75015 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:28.200846 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32560c47-5cd9-443e-a349-83650f4246b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:28.201398 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32560c47-5cd9-443e-a349-83650f4246b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:28.201702 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32560c47-5cd9-443e-a349-83650f4246b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:28.201975 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32560c47-5cd9-443e-a349-83650f4246b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:28.202624 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32560c47-5cd9-443e-a349-83650f4246b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:38:28.202928 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32560c47-5cd9-443e-a349-83650f4246b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:28.204527 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32560c47-5cd9-443e-a349-83650f4246b2 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:28.221675 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 782/1542] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 15 07:38:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:38:28.226189 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb948db-167b-4d8f-96af-ed7cf78f4634 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:28.239018 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb948db-167b-4d8f-96af-ed7cf78f4634 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:28.247255 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2893fd-caa2-497e-af01-7f8ca10b9a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:28.247893 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2893fd-caa2-497e-af01-7f8ca10b9a05 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:28.248175 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2893fd-caa2-497e-af01-7f8ca10b9a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:28.248439 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2893fd-caa2-497e-af01-7f8ca10b9a05 admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:28.249091 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2893fd-caa2-497e-af01-7f8ca10b9a05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:28.251017 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2893fd-caa2-497e-af01-7f8ca10b9a05 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:28.407712 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 761/1543] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:38:28 2022] POST /identity/v3/projects => generated 379 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 07:38:28.412191 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32560c47-5cd9-443e-a349-83650f4246b2 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:28.428254 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32560c47-5cd9-443e-a349-83650f4246b2 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:28.506802 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7dcd8e-26df-4f6e-aade-04562a534f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:28.507334 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7dcd8e-26df-4f6e-aade-04562a534f0f admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:28.507610 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7dcd8e-26df-4f6e-aade-04562a534f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:28.507880 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7dcd8e-26df-4f6e-aade-04562a534f0f admin admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:28.508541 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7dcd8e-26df-4f6e-aade-04562a534f0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:28.510506 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7dcd8e-26df-4f6e-aade-04562a534f0f admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:28.571647 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4412f1-0aa4-4571-aa46-184d21e74210 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:28.572046 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4412f1-0aa4-4571-aa46-184d21e74210 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:28.572241 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4412f1-0aa4-4571-aa46-184d21e74210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:28.602294 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-3b4412f1-0aa4-4571-aa46-184d21e74210 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:38:28.607574 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-3b4412f1-0aa4-4571-aa46-184d21e74210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:28.622910 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 762/1544] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:38:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 07:38:28.624356 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24420b7-e157-451c-b0e5-61e9f60613c1 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:28.624645 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24420b7-e157-451c-b0e5-61e9f60613c1 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:28.624914 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24420b7-e157-451c-b0e5-61e9f60613c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:28.640797 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 783/1545] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:38:28 2022] POST /identity/v3/users => generated 290 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:38:28.642152 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73097bd2-e0e1-4da2-9d6a-8cc01526f105 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:28.642441 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73097bd2-e0e1-4da2-9d6a-8cc01526f105 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:28.642714 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73097bd2-e0e1-4da2-9d6a-8cc01526f105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:28.695132 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-c24420b7-e157-451c-b0e5-61e9f60613c1 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:38:28.702059 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-c24420b7-e157-451c-b0e5-61e9f60613c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:28.705287 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-73097bd2-e0e1-4da2-9d6a-8cc01526f105 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:38:28.708626 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-73097bd2-e0e1-4da2-9d6a-8cc01526f105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:28.729912 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 763/1546] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:38:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:38:28.738932 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 784/1547] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:38:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:38:28.745131 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24420b7-e157-451c-b0e5-61e9f60613c1 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:28.753480 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e733d233-500b-404b-8afd-064962addf04 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:28.753800 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e733d233-500b-404b-8afd-064962addf04 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:28.754104 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e733d233-500b-404b-8afd-064962addf04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:28.761865 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24420b7-e157-451c-b0e5-61e9f60613c1 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:28.767941 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e98a97-a29d-4616-9dc2-771466168b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:28.768893 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e98a97-a29d-4616-9dc2-771466168b78 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:28.769719 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e98a97-a29d-4616-9dc2-771466168b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:28.770192 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e98a97-a29d-4616-9dc2-771466168b78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:28.771721 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e98a97-a29d-4616-9dc2-771466168b78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:28.774119 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e98a97-a29d-4616-9dc2-771466168b78 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:28.787919 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 764/1548] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:38:28 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:38:28.788775 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e92427-43f9-420a-a48f-9f8aa672e13a None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:28.789025 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e92427-43f9-420a-a48f-9f8aa672e13a None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:28.790261 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e92427-43f9-420a-a48f-9f8aa672e13a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:28.805823 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-e733d233-500b-404b-8afd-064962addf04 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:38:28.807074 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-e733d233-500b-404b-8afd-064962addf04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:28.816596 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 785/1549] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:38:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:38:28.818519 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1890486e-fc15-48ec-8253-5fb1f28c6262 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:28.818934 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1890486e-fc15-48ec-8253-5fb1f28c6262 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:28.819430 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1890486e-fc15-48ec-8253-5fb1f28c6262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:28.821463 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-13e92427-43f9-420a-a48f-9f8aa672e13a None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:38:28.823175 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-13e92427-43f9-420a-a48f-9f8aa672e13a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:28.835217 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 765/1550] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:38:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:38:28.836287 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e92427-43f9-420a-a48f-9f8aa672e13a None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:28.843293 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-1890486e-fc15-48ec-8253-5fb1f28c6262 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:38:28.845895 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-1890486e-fc15-48ec-8253-5fb1f28c6262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:28.848560 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e92427-43f9-420a-a48f-9f8aa672e13a None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:28.854122 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096119b7-3a39-48e3-8a16-f30b21179a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:28.859277 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096119b7-3a39-48e3-8a16-f30b21179a0b admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:28.859277 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096119b7-3a39-48e3-8a16-f30b21179a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:28.859277 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096119b7-3a39-48e3-8a16-f30b21179a0b admin admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:28.859277 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096119b7-3a39-48e3-8a16-f30b21179a0b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:38:28.859277 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096119b7-3a39-48e3-8a16-f30b21179a0b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:28.859701 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: WARNING py.warnings [None req-096119b7-3a39-48e3-8a16-f30b21179a0b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 15 07:38:28.859701 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: warnings.warn(msg) Sep 15 07:38:28.859701 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: Sep 15 07:38:28.859701 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096119b7-3a39-48e3-8a16-f30b21179a0b admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:28.868689 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 786/1551] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:38:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:38:28.870218 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244a42bf-7672-4420-af0f-0cebb667c224 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:28.870509 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244a42bf-7672-4420-af0f-0cebb667c224 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:28.870829 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244a42bf-7672-4420-af0f-0cebb667c224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:28.876554 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 766/1552] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 15 07:38:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:38:28.877462 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c3333d-0dc3-41cc-91f1-dc498de95281 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:28.878033 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c3333d-0dc3-41cc-91f1-dc498de95281 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:28.878186 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c3333d-0dc3-41cc-91f1-dc498de95281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:28.899357 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-244a42bf-7672-4420-af0f-0cebb667c224 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:38:28.900196 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-64c3333d-0dc3-41cc-91f1-dc498de95281 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:38:28.900563 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-244a42bf-7672-4420-af0f-0cebb667c224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:28.902330 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-64c3333d-0dc3-41cc-91f1-dc498de95281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:28.915069 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 767/1553] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:38:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:38:28.916075 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c3333d-0dc3-41cc-91f1-dc498de95281 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:28.926709 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c3333d-0dc3-41cc-91f1-dc498de95281 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:28.927142 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 787/1554] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:38:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:38:28.929637 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70167d9-23a6-45e7-8e4d-490555d5f6e2 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:28.930099 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70167d9-23a6-45e7-8e4d-490555d5f6e2 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:28.930226 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70167d9-23a6-45e7-8e4d-490555d5f6e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:28.932110 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d1553e-e571-4631-b87e-ecfeab9d81cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:28.932730 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d1553e-e571-4631-b87e-ecfeab9d81cd admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:28.933015 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d1553e-e571-4631-b87e-ecfeab9d81cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:28.933292 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d1553e-e571-4631-b87e-ecfeab9d81cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:28.934140 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d1553e-e571-4631-b87e-ecfeab9d81cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:28.935764 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d1553e-e571-4631-b87e-ecfeab9d81cd admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:28.952592 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-d70167d9-23a6-45e7-8e4d-490555d5f6e2 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:38:28.953857 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-d70167d9-23a6-45e7-8e4d-490555d5f6e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:28.954593 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 768/1555] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:38:28 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:38:28.955965 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37031514-1bb5-424f-b14b-f8b2de047a8d None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:28.956263 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37031514-1bb5-424f-b14b-f8b2de047a8d None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:28.956546 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37031514-1bb5-424f-b14b-f8b2de047a8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:28.965697 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 788/1556] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:38:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:38:28.965697 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cefb30-4928-47fa-9043-e604fc341305 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:28.965697 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cefb30-4928-47fa-9043-e604fc341305 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:28.965697 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cefb30-4928-47fa-9043-e604fc341305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:28.980952 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-37031514-1bb5-424f-b14b-f8b2de047a8d None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:38:28.982201 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-37031514-1bb5-424f-b14b-f8b2de047a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:28.984359 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-16cefb30-4928-47fa-9043-e604fc341305 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:38:28.985846 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-16cefb30-4928-47fa-9043-e604fc341305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:28.991039 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 769/1557] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:38:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:38:28.997096 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37031514-1bb5-424f-b14b-f8b2de047a8d None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:28.997210 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 789/1558] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:38:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:38:28.997210 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cefb30-4928-47fa-9043-e604fc341305 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:29.006196 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cefb30-4928-47fa-9043-e604fc341305 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:29.006420 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37031514-1bb5-424f-b14b-f8b2de047a8d None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:29.011715 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240b2651-8b3b-47ae-aac6-ac3731e6b842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:29.042354 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b2651-8b3b-47ae-aac6-ac3731e6b842 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:29.042354 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b2651-8b3b-47ae-aac6-ac3731e6b842 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:29.042354 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b2651-8b3b-47ae-aac6-ac3731e6b842 admin admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:29.042354 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240b2651-8b3b-47ae-aac6-ac3731e6b842 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:38:29.042906 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a175720-04a0-4dbb-9e29-7eb03564bf12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:29.043043 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240b2651-8b3b-47ae-aac6-ac3731e6b842 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:29.043142 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a175720-04a0-4dbb-9e29-7eb03564bf12 admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:29.043142 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a175720-04a0-4dbb-9e29-7eb03564bf12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:29.043142 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a175720-04a0-4dbb-9e29-7eb03564bf12 admin admin] PATH_INFO: `/v3/projects` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:29.043142 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a175720-04a0-4dbb-9e29-7eb03564bf12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:29.043531 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240b2651-8b3b-47ae-aac6-ac3731e6b842 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:29.043658 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: WARNING py.warnings [None req-6a175720-04a0-4dbb-9e29-7eb03564bf12 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 15 07:38:29.043658 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: warnings.warn(msg) Sep 15 07:38:29.043658 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: Sep 15 07:38:29.043658 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a175720-04a0-4dbb-9e29-7eb03564bf12 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:29.043996 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 770/1559] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 15 07:38:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:38:29.043996 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82b0dd1-b1bd-45a2-a80a-98bddd9fe853 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:29.043996 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82b0dd1-b1bd-45a2-a80a-98bddd9fe853 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:29.043996 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82b0dd1-b1bd-45a2-a80a-98bddd9fe853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:29.057011 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-b82b0dd1-b1bd-45a2-a80a-98bddd9fe853 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:38:29.058311 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-b82b0dd1-b1bd-45a2-a80a-98bddd9fe853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:29.067085 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 771/1560] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:38:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:38:29.068442 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1091a181-03db-4c76-b9ca-8b5a6e65e7d2 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:29.068737 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1091a181-03db-4c76-b9ca-8b5a6e65e7d2 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:29.069011 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1091a181-03db-4c76-b9ca-8b5a6e65e7d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:29.089718 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-1091a181-03db-4c76-b9ca-8b5a6e65e7d2 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:38:29.090926 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-1091a181-03db-4c76-b9ca-8b5a6e65e7d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:29.098812 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 772/1561] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:38:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:38:29.100415 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1091a181-03db-4c76-b9ca-8b5a6e65e7d2 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:29.109791 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1091a181-03db-4c76-b9ca-8b5a6e65e7d2 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:29.114291 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a6471d-4c00-45ee-a76d-f8a31464e088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:29.114948 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a6471d-4c00-45ee-a76d-f8a31464e088 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:29.115229 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a6471d-4c00-45ee-a76d-f8a31464e088 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:29.115510 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a6471d-4c00-45ee-a76d-f8a31464e088 admin admin] PATH_INFO: `/v3/projects/ac0ec290949c4a41ba97f0efc53628d4/users/dedde243557c450188a0b7cdf9f7b1e7/roles/053e12af84e943e699460a9d76c2072b` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:29.130770 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a6471d-4c00-45ee-a76d-f8a31464e088 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac0ec290949c4a41ba97f0efc53628d4, user_id=dedde243557c450188a0b7cdf9f7b1e7, role_id=053e12af84e943e699460a9d76c2072b)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:29.133462 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a6471d-4c00-45ee-a76d-f8a31464e088 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:29.161195 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 790/1562] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:38:28 2022] POST /identity/v3/projects => generated 381 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:38:29.162983 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a175720-04a0-4dbb-9e29-7eb03564bf12 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:29.224801 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a175720-04a0-4dbb-9e29-7eb03564bf12 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:29.229713 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0ba074-00c5-470f-bdf9-44a3380ca258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:29.230427 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0ba074-00c5-470f-bdf9-44a3380ca258 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:29.230706 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0ba074-00c5-470f-bdf9-44a3380ca258 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:29.231084 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0ba074-00c5-470f-bdf9-44a3380ca258 admin admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:29.231733 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0ba074-00c5-470f-bdf9-44a3380ca258 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:38:29.232038 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0ba074-00c5-470f-bdf9-44a3380ca258 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:29.233772 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0ba074-00c5-470f-bdf9-44a3380ca258 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:29.234143 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 773/1563] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:38:29 2022] PUT /identity/v3/projects/ac0ec290949c4a41ba97f0efc53628d4/users/dedde243557c450188a0b7cdf9f7b1e7/roles/053e12af84e943e699460a9d76c2072b => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:38:29.237884 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a6471d-4c00-45ee-a76d-f8a31464e088 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:29.259429 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 791/1564] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 15 07:38:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:38:29.259429 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9110bee9-310b-4bfa-b664-c8c3c32f6d3b None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:29.259429 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9110bee9-310b-4bfa-b664-c8c3c32f6d3b None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:29.259429 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9110bee9-310b-4bfa-b664-c8c3c32f6d3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:29.261811 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:38:29.276358 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-9110bee9-310b-4bfa-b664-c8c3c32f6d3b None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:38:29.278017 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-9110bee9-310b-4bfa-b664-c8c3c32f6d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:29.297692 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a6471d-4c00-45ee-a76d-f8a31464e088 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:29.302206 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9705964b-ab67-4be4-b122-e98ff7c79e82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:29.302835 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9705964b-ab67-4be4-b122-e98ff7c79e82 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:29.303115 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9705964b-ab67-4be4-b122-e98ff7c79e82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:29.303386 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9705964b-ab67-4be4-b122-e98ff7c79e82 admin admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:29.304027 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9705964b-ab67-4be4-b122-e98ff7c79e82 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:38:29.304333 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9705964b-ab67-4be4-b122-e98ff7c79e82 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:29.305938 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9705964b-ab67-4be4-b122-e98ff7c79e82 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:29.312906 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:38:29.317783 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 774/1565] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 15 07:38:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:38:29.319194 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9705964b-ab67-4be4-b122-e98ff7c79e82 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:29.328970 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9705964b-ab67-4be4-b122-e98ff7c79e82 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:29.333804 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8151eeb-424e-4949-b45f-8f0445ae1f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:29.336873 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8151eeb-424e-4949-b45f-8f0445ae1f0f admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:29.336873 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8151eeb-424e-4949-b45f-8f0445ae1f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:29.336873 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8151eeb-424e-4949-b45f-8f0445ae1f0f admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:29.336873 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8151eeb-424e-4949-b45f-8f0445ae1f0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:29.337528 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8151eeb-424e-4949-b45f-8f0445ae1f0f admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:29.338531 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 792/1566] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:38:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:38:29.339483 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9110bee9-310b-4bfa-b664-c8c3c32f6d3b None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:29.351590 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9110bee9-310b-4bfa-b664-c8c3c32f6d3b None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:29.356291 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e359539d-b936-40e6-aece-32c7a0e6d9c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:29.359860 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e359539d-b936-40e6-aece-32c7a0e6d9c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:29.359860 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e359539d-b936-40e6-aece-32c7a0e6d9c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:29.359860 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e359539d-b936-40e6-aece-32c7a0e6d9c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:29.359860 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e359539d-b936-40e6-aece-32c7a0e6d9c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:38:29.359860 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e359539d-b936-40e6-aece-32c7a0e6d9c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:29.360323 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e359539d-b936-40e6-aece-32c7a0e6d9c7 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:29.371659 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 793/1567] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 15 07:38:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:38:29.372594 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e359539d-b936-40e6-aece-32c7a0e6d9c7 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:29.383270 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e359539d-b936-40e6-aece-32c7a0e6d9c7 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:29.389251 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec06e74-b063-4724-90c3-8b0ff23d7a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:29.389871 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec06e74-b063-4724-90c3-8b0ff23d7a0b admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:29.390152 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec06e74-b063-4724-90c3-8b0ff23d7a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:29.390503 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec06e74-b063-4724-90c3-8b0ff23d7a0b admin admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:29.391137 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec06e74-b063-4724-90c3-8b0ff23d7a0b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:38:29.391438 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec06e74-b063-4724-90c3-8b0ff23d7a0b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:29.393038 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec06e74-b063-4724-90c3-8b0ff23d7a0b admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:29.406927 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 794/1568] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 15 07:38:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:38:29.407706 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec06e74-b063-4724-90c3-8b0ff23d7a0b admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:29.417681 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec06e74-b063-4724-90c3-8b0ff23d7a0b admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:29.422432 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e13e286-797f-4142-9547-21b8687e2349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:29.425733 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e13e286-797f-4142-9547-21b8687e2349 admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:29.425733 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e13e286-797f-4142-9547-21b8687e2349 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:29.425733 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e13e286-797f-4142-9547-21b8687e2349 admin admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:29.425733 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e13e286-797f-4142-9547-21b8687e2349 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:29.426113 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e13e286-797f-4142-9547-21b8687e2349 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:29.456962 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 775/1569] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:38:29 2022] POST /identity/v3/projects => generated 385 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:38:29.458627 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa210774-3f62-4a85-892e-a49781de4c3b None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:29.459914 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa210774-3f62-4a85-892e-a49781de4c3b None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:29.460392 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa210774-3f62-4a85-892e-a49781de4c3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:29.490702 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-fa210774-3f62-4a85-892e-a49781de4c3b None None] MFA Rules not processed for user `dedde243557c450188a0b7cdf9f7b1e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:38:29.491899 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-fa210774-3f62-4a85-892e-a49781de4c3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:29.537873 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 795/1570] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:38:29 2022] POST /identity/v3/users => generated 291 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:38:29.539282 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e13e286-797f-4142-9547-21b8687e2349 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:29.550406 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e13e286-797f-4142-9547-21b8687e2349 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:29.567364 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 776/1571] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:38:29 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:38:29.568846 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa210774-3f62-4a85-892e-a49781de4c3b None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:29.616138 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e62877e-e872-4335-9b3c-879e3688f5e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:29.616924 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e62877e-e872-4335-9b3c-879e3688f5e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:29.617204 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e62877e-e872-4335-9b3c-879e3688f5e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:29.617554 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e62877e-e872-4335-9b3c-879e3688f5e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:29.618283 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e62877e-e872-4335-9b3c-879e3688f5e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:29.620419 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e62877e-e872-4335-9b3c-879e3688f5e6 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:29.639858 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa210774-3f62-4a85-892e-a49781de4c3b None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:29.645502 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdbb425-67aa-4b70-a431-5fd0355f8b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:29.646188 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdbb425-67aa-4b70-a431-5fd0355f8b8c admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:29.646456 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdbb425-67aa-4b70-a431-5fd0355f8b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:29.646731 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdbb425-67aa-4b70-a431-5fd0355f8b8c admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:29.647385 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdbb425-67aa-4b70-a431-5fd0355f8b8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:29.649444 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdbb425-67aa-4b70-a431-5fd0355f8b8c admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:29.716189 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 796/1572] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:38:29 2022] POST /identity/v3/projects => generated 385 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:38:29.717137 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e62877e-e872-4335-9b3c-879e3688f5e6 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:29.756636 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 777/1573] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:38:29 2022] POST /identity/v3/projects => generated 379 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:38:29.758508 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdbb425-67aa-4b70-a431-5fd0355f8b8c admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:29.775133 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:38:29.797127 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e62877e-e872-4335-9b3c-879e3688f5e6 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:29.823795 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:38:29.839340 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdbb425-67aa-4b70-a431-5fd0355f8b8c admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:29.846503 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a3f8e9-5da2-44dc-8e4f-65fa8addee2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:29.848619 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a3f8e9-5da2-44dc-8e4f-65fa8addee2d admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:29.848619 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a3f8e9-5da2-44dc-8e4f-65fa8addee2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:29.848619 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a3f8e9-5da2-44dc-8e4f-65fa8addee2d admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:29.848619 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a3f8e9-5da2-44dc-8e4f-65fa8addee2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:29.850088 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a3f8e9-5da2-44dc-8e4f-65fa8addee2d admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:29.866583 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338ebd27-3ed8-4e4a-af88-9ba375d2a6e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:29.867198 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338ebd27-3ed8-4e4a-af88-9ba375d2a6e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:29.867477 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338ebd27-3ed8-4e4a-af88-9ba375d2a6e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:29.867782 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338ebd27-3ed8-4e4a-af88-9ba375d2a6e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:29.868437 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338ebd27-3ed8-4e4a-af88-9ba375d2a6e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:38:29.868741 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338ebd27-3ed8-4e4a-af88-9ba375d2a6e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:29.870482 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338ebd27-3ed8-4e4a-af88-9ba375d2a6e6 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:29.883519 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 797/1574] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 15 07:38:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:38:29.884592 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338ebd27-3ed8-4e4a-af88-9ba375d2a6e6 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:29.895290 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338ebd27-3ed8-4e4a-af88-9ba375d2a6e6 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:29.902563 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2736af6-5301-4ccc-8de8-2da81a181764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:29.903206 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2736af6-5301-4ccc-8de8-2da81a181764 admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:29.907298 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2736af6-5301-4ccc-8de8-2da81a181764 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:29.907298 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2736af6-5301-4ccc-8de8-2da81a181764 admin admin] PATH_INFO: `/v3/projects` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:29.907298 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2736af6-5301-4ccc-8de8-2da81a181764 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:29.907298 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2736af6-5301-4ccc-8de8-2da81a181764 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:29.958200 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 778/1575] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:38:29 2022] POST /identity/v3/projects => generated 371 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:38:29.959640 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a3f8e9-5da2-44dc-8e4f-65fa8addee2d admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:30.008408 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 798/1576] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:38:29 2022] POST /identity/v3/projects => generated 389 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:38:30.009312 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2736af6-5301-4ccc-8de8-2da81a181764 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:30.056370 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a3f8e9-5da2-44dc-8e4f-65fa8addee2d admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:30.080826 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2736af6-5301-4ccc-8de8-2da81a181764 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:30.086606 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a7cbcc-b60d-4a90-8fe6-cc40e992181a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:30.087277 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a7cbcc-b60d-4a90-8fe6-cc40e992181a admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:30.087554 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a7cbcc-b60d-4a90-8fe6-cc40e992181a admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:30.087854 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a7cbcc-b60d-4a90-8fe6-cc40e992181a admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:30.088724 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a7cbcc-b60d-4a90-8fe6-cc40e992181a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:30.090378 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a7cbcc-b60d-4a90-8fe6-cc40e992181a admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:30.106009 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 799/1577] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:38:30 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:38:30.107346 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a7cbcc-b60d-4a90-8fe6-cc40e992181a admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:30.123166 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a7cbcc-b60d-4a90-8fe6-cc40e992181a admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:30.128525 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c08c364-eb3b-4c9c-8c39-72f06755aeca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:30.129126 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c08c364-eb3b-4c9c-8c39-72f06755aeca admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:30.129405 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c08c364-eb3b-4c9c-8c39-72f06755aeca admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:30.129723 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c08c364-eb3b-4c9c-8c39-72f06755aeca admin admin] PATH_INFO: `/v3/projects` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:30.130357 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c08c364-eb3b-4c9c-8c39-72f06755aeca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:30.132785 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c08c364-eb3b-4c9c-8c39-72f06755aeca admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:30.134310 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c64bf0c-4290-463d-af1d-9bae348ff0e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:30.135263 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c64bf0c-4290-463d-af1d-9bae348ff0e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:30.135747 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c64bf0c-4290-463d-af1d-9bae348ff0e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:30.136194 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c64bf0c-4290-463d-af1d-9bae348ff0e7 admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:30.137264 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c64bf0c-4290-463d-af1d-9bae348ff0e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:30.139061 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: WARNING py.warnings [None req-6c64bf0c-4290-463d-af1d-9bae348ff0e7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 15 07:38:30.139061 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: warnings.warn(msg) Sep 15 07:38:30.139061 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: Sep 15 07:38:30.140282 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c64bf0c-4290-463d-af1d-9bae348ff0e7 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:30.247359 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 779/1578] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:38:29 2022] POST /identity/v3/users => generated 293 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:38:30.250400 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c64bf0c-4290-463d-af1d-9bae348ff0e7 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:30.265660 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c64bf0c-4290-463d-af1d-9bae348ff0e7 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:30.273934 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 800/1579] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:38:30 2022] POST /identity/v3/projects => generated 379 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:38:30.274444 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3537c284-814e-4274-b28a-db3e6e8863bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:30.274579 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c08c364-eb3b-4c9c-8c39-72f06755aeca admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:30.274690 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3537c284-814e-4274-b28a-db3e6e8863bc admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:30.274690 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3537c284-814e-4274-b28a-db3e6e8863bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:30.274690 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3537c284-814e-4274-b28a-db3e6e8863bc admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:30.274690 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3537c284-814e-4274-b28a-db3e6e8863bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:30.275093 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3537c284-814e-4274-b28a-db3e6e8863bc admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:30.322878 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:38:30.352440 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c08c364-eb3b-4c9c-8c39-72f06755aeca admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:30.359637 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820ad116-370f-457f-be31-015166839eff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:30.360670 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820ad116-370f-457f-be31-015166839eff admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:30.361147 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820ad116-370f-457f-be31-015166839eff admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:30.361604 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820ad116-370f-457f-be31-015166839eff admin admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:30.364207 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820ad116-370f-457f-be31-015166839eff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:30.367834 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820ad116-370f-457f-be31-015166839eff admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:30.377522 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 780/1580] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:38:30 2022] POST /identity/v3/users => generated 293 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:38:30.378702 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3537c284-814e-4274-b28a-db3e6e8863bc admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:30.391151 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3537c284-814e-4274-b28a-db3e6e8863bc admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:30.399189 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3e8a11-5616-48f9-bcc7-170a388be1aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:30.400160 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3e8a11-5616-48f9-bcc7-170a388be1aa admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:30.401898 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3e8a11-5616-48f9-bcc7-170a388be1aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:30.401898 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3e8a11-5616-48f9-bcc7-170a388be1aa admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:30.402353 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3e8a11-5616-48f9-bcc7-170a388be1aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:30.405373 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3e8a11-5616-48f9-bcc7-170a388be1aa admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:30.457720 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 801/1581] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:38:30 2022] POST /identity/v3/users => generated 290 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:38:30.458548 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820ad116-370f-457f-be31-015166839eff admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:30.470200 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820ad116-370f-457f-be31-015166839eff admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:30.476766 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eed034-d413-4665-b106-b85f4899fe3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:30.477376 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eed034-d413-4665-b106-b85f4899fe3a admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:30.477698 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eed034-d413-4665-b106-b85f4899fe3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:30.477977 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eed034-d413-4665-b106-b85f4899fe3a admin admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:30.478618 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1eed034-d413-4665-b106-b85f4899fe3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:30.480574 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1eed034-d413-4665-b106-b85f4899fe3a admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:30.533033 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 781/1582] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:38:30 2022] POST /identity/v3/projects => generated 405 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:38:30.533944 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3e8a11-5616-48f9-bcc7-170a388be1aa admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:30.571028 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:38:30.597508 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3e8a11-5616-48f9-bcc7-170a388be1aa admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:30.602177 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646c9a8e-a629-4058-aa42-5e36acfffd0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:30.602834 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c9a8e-a629-4058-aa42-5e36acfffd0b admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:30.603060 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c9a8e-a629-4058-aa42-5e36acfffd0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:30.603330 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c9a8e-a629-4058-aa42-5e36acfffd0b admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:30.604009 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646c9a8e-a629-4058-aa42-5e36acfffd0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:30.605942 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646c9a8e-a629-4058-aa42-5e36acfffd0b admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:30.619585 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 802/1583] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:38:30 2022] POST /identity/v3/users => generated 286 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:38:30.619585 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eed034-d413-4665-b106-b85f4899fe3a admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:30.627988 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eed034-d413-4665-b106-b85f4899fe3a admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:30.633276 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c6ff56-5608-441d-a3c0-621799c45e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:30.634090 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c6ff56-5608-441d-a3c0-621799c45e73 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:30.634090 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c6ff56-5608-441d-a3c0-621799c45e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:30.635641 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c6ff56-5608-441d-a3c0-621799c45e73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:30.635641 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c6ff56-5608-441d-a3c0-621799c45e73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:30.636621 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c6ff56-5608-441d-a3c0-621799c45e73 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:30.647576 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 803/1584] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:38:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:38:30.648436 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c6ff56-5608-441d-a3c0-621799c45e73 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:30.657535 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c6ff56-5608-441d-a3c0-621799c45e73 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:30.662114 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaee5ec1-72fb-496c-891a-cc6f71d6c514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:30.662762 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaee5ec1-72fb-496c-891a-cc6f71d6c514 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:30.663039 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaee5ec1-72fb-496c-891a-cc6f71d6c514 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:30.663310 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaee5ec1-72fb-496c-891a-cc6f71d6c514 admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:30.663922 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaee5ec1-72fb-496c-891a-cc6f71d6c514 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:30.665466 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaee5ec1-72fb-496c-891a-cc6f71d6c514 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:30.675101 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 804/1585] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:38:30 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:38:30.675838 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaee5ec1-72fb-496c-891a-cc6f71d6c514 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:30.685759 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 782/1586] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:38:30 2022] POST /identity/v3/users => generated 295 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:38:30.685759 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646c9a8e-a629-4058-aa42-5e36acfffd0b admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:30.691349 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaee5ec1-72fb-496c-891a-cc6f71d6c514 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:30.691349 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc8ca73-87c2-4fd4-9375-fcdce9aefaed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:30.691597 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc8ca73-87c2-4fd4-9375-fcdce9aefaed admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:30.691877 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc8ca73-87c2-4fd4-9375-fcdce9aefaed admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:30.692180 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc8ca73-87c2-4fd4-9375-fcdce9aefaed admin admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:30.692874 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc8ca73-87c2-4fd4-9375-fcdce9aefaed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:30.695684 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc8ca73-87c2-4fd4-9375-fcdce9aefaed admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:30.697795 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646c9a8e-a629-4058-aa42-5e36acfffd0b admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:30.702546 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d75bc33-a848-44ec-93bc-c983a3831e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:30.703144 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d75bc33-a848-44ec-93bc-c983a3831e5c admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:30.703449 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d75bc33-a848-44ec-93bc-c983a3831e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:30.703936 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d75bc33-a848-44ec-93bc-c983a3831e5c admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:30.705027 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d75bc33-a848-44ec-93bc-c983a3831e5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:30.707165 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d75bc33-a848-44ec-93bc-c983a3831e5c admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:30.717984 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 783/1587] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:38:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:38:30.718801 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d75bc33-a848-44ec-93bc-c983a3831e5c admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:30.729228 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d75bc33-a848-44ec-93bc-c983a3831e5c admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:30.733890 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7f5bd0-465f-4c8c-bc4a-7eb4135f8af4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:30.734494 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7f5bd0-465f-4c8c-bc4a-7eb4135f8af4 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:30.734779 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7f5bd0-465f-4c8c-bc4a-7eb4135f8af4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:30.735055 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7f5bd0-465f-4c8c-bc4a-7eb4135f8af4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:30.735692 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7f5bd0-465f-4c8c-bc4a-7eb4135f8af4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:30.737298 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7f5bd0-465f-4c8c-bc4a-7eb4135f8af4 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:30.748885 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 784/1588] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:38:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:38:30.749853 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7f5bd0-465f-4c8c-bc4a-7eb4135f8af4 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:30.759198 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7f5bd0-465f-4c8c-bc4a-7eb4135f8af4 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:30.763887 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea024348-29a5-49e4-aa92-11da9c00658a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:30.764464 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea024348-29a5-49e4-aa92-11da9c00658a admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:30.764751 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea024348-29a5-49e4-aa92-11da9c00658a admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:30.765030 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea024348-29a5-49e4-aa92-11da9c00658a admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:30.765692 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea024348-29a5-49e4-aa92-11da9c00658a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:30.767574 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea024348-29a5-49e4-aa92-11da9c00658a admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:30.801822 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 805/1589] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:38:30 2022] POST /identity/v3/users => generated 296 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:38:30.801822 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc8ca73-87c2-4fd4-9375-fcdce9aefaed admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:30.810525 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc8ca73-87c2-4fd4-9375-fcdce9aefaed admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:30.815159 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbebfee0-0556-496f-81db-1806267a906d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:30.815761 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbebfee0-0556-496f-81db-1806267a906d admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:30.816044 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbebfee0-0556-496f-81db-1806267a906d admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:30.816320 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbebfee0-0556-496f-81db-1806267a906d admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:30.816967 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbebfee0-0556-496f-81db-1806267a906d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:30.818652 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbebfee0-0556-496f-81db-1806267a906d admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:30.829260 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 806/1590] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:38:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:38:30.840344 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbebfee0-0556-496f-81db-1806267a906d admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:30.840344 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbebfee0-0556-496f-81db-1806267a906d admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:30.844249 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dee87a-23a5-4429-b509-f6632babae62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:30.844878 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dee87a-23a5-4429-b509-f6632babae62 admin admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:30.845165 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dee87a-23a5-4429-b509-f6632babae62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:30.845442 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dee87a-23a5-4429-b509-f6632babae62 admin admin] PATH_INFO: `/v3/projects/166bf7277833431ebc3e8abc17f10c0d/users/96bbf934e49b4bff9810a8e11c55b49b/roles/053e12af84e943e699460a9d76c2072b` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:30.866879 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dee87a-23a5-4429-b509-f6632babae62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=166bf7277833431ebc3e8abc17f10c0d, user_id=96bbf934e49b4bff9810a8e11c55b49b, role_id=053e12af84e943e699460a9d76c2072b)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:30.866879 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: WARNING py.warnings [None req-13dee87a-23a5-4429-b509-f6632babae62 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 15 07:38:30.866879 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: warnings.warn(msg) Sep 15 07:38:30.866879 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: Sep 15 07:38:30.867318 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 785/1591] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:38:30 2022] POST /identity/v3/users => generated 303 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:38:30.867318 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea024348-29a5-49e4-aa92-11da9c00658a admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:30.867510 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dee87a-23a5-4429-b509-f6632babae62 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:30.875429 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea024348-29a5-49e4-aa92-11da9c00658a admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:30.880221 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54c952b-0c59-4b89-98ef-916a52786e06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:30.881266 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54c952b-0c59-4b89-98ef-916a52786e06 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:30.881550 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54c952b-0c59-4b89-98ef-916a52786e06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:30.881854 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54c952b-0c59-4b89-98ef-916a52786e06 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:30.882525 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54c952b-0c59-4b89-98ef-916a52786e06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:30.884114 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54c952b-0c59-4b89-98ef-916a52786e06 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:30.896319 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 786/1592] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:38:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:38:30.896319 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54c952b-0c59-4b89-98ef-916a52786e06 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:30.904751 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54c952b-0c59-4b89-98ef-916a52786e06 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:30.909263 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1791cd-b74c-4494-94cb-98e68890f5b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:30.909951 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1791cd-b74c-4494-94cb-98e68890f5b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:30.910228 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1791cd-b74c-4494-94cb-98e68890f5b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:30.910498 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1791cd-b74c-4494-94cb-98e68890f5b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:30.911125 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1791cd-b74c-4494-94cb-98e68890f5b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:30.913283 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1791cd-b74c-4494-94cb-98e68890f5b3 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:30.923826 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 787/1593] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:38:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:38:30.924590 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1791cd-b74c-4494-94cb-98e68890f5b3 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:30.934680 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1791cd-b74c-4494-94cb-98e68890f5b3 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:30.942223 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f209eb0-576c-4b13-9200-0d38e0ed757c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:30.942814 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f209eb0-576c-4b13-9200-0d38e0ed757c admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:30.943221 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f209eb0-576c-4b13-9200-0d38e0ed757c admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:30.943348 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 807/1594] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:38:30 2022] PUT /identity/v3/projects/166bf7277833431ebc3e8abc17f10c0d/users/96bbf934e49b4bff9810a8e11c55b49b/roles/053e12af84e943e699460a9d76c2072b => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:38:30.943563 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f209eb0-576c-4b13-9200-0d38e0ed757c admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:30.944294 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f209eb0-576c-4b13-9200-0d38e0ed757c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:30.944635 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dee87a-23a5-4429-b509-f6632babae62 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:30.945929 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f209eb0-576c-4b13-9200-0d38e0ed757c admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:30.963588 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 788/1595] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:38:30 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:38:30.967506 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f209eb0-576c-4b13-9200-0d38e0ed757c admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:30.978359 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f209eb0-576c-4b13-9200-0d38e0ed757c admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:31.015404 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dee87a-23a5-4429-b509-f6632babae62 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:31.020310 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358623d6-7000-41ba-b6df-cb5a48ee46a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:31.020921 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358623d6-7000-41ba-b6df-cb5a48ee46a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:31.021226 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358623d6-7000-41ba-b6df-cb5a48ee46a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:31.021906 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358623d6-7000-41ba-b6df-cb5a48ee46a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:31.022161 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358623d6-7000-41ba-b6df-cb5a48ee46a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:31.023762 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358623d6-7000-41ba-b6df-cb5a48ee46a5 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:31.037419 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 808/1596] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:38:30 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:38:31.037419 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358623d6-7000-41ba-b6df-cb5a48ee46a5 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:31.042048 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a019065c-c6fc-4c99-ad3a-e24804da227c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:31.042690 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a019065c-c6fc-4c99-ad3a-e24804da227c admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:31.043005 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a019065c-c6fc-4c99-ad3a-e24804da227c admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:31.043280 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a019065c-c6fc-4c99-ad3a-e24804da227c admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:31.043913 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a019065c-c6fc-4c99-ad3a-e24804da227c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:31.045540 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a019065c-c6fc-4c99-ad3a-e24804da227c admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:31.047250 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358623d6-7000-41ba-b6df-cb5a48ee46a5 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:31.051774 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e9c041-07c2-47de-8a3d-e5bda92a9050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:31.052350 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e9c041-07c2-47de-8a3d-e5bda92a9050 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:31.053218 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e9c041-07c2-47de-8a3d-e5bda92a9050 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:31.053522 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e9c041-07c2-47de-8a3d-e5bda92a9050 admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:31.054311 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e9c041-07c2-47de-8a3d-e5bda92a9050 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:31.055862 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e9c041-07c2-47de-8a3d-e5bda92a9050 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:31.059791 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 789/1597] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:38:30 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:38:31.059791 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a019065c-c6fc-4c99-ad3a-e24804da227c admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:31.069267 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 809/1598] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:38:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:38:31.069267 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e9c041-07c2-47de-8a3d-e5bda92a9050 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:31.071981 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a019065c-c6fc-4c99-ad3a-e24804da227c admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:31.075925 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e9c041-07c2-47de-8a3d-e5bda92a9050 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:31.076709 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc331608-766f-4e13-951b-7def57e1eebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:31.077371 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc331608-766f-4e13-951b-7def57e1eebd admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:31.077771 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc331608-766f-4e13-951b-7def57e1eebd admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:31.078117 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc331608-766f-4e13-951b-7def57e1eebd admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:31.078821 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc331608-766f-4e13-951b-7def57e1eebd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:31.080387 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc331608-766f-4e13-951b-7def57e1eebd admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:31.080512 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18f1a67-628e-4fd7-8990-e84eab299d07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:31.081036 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f1a67-628e-4fd7-8990-e84eab299d07 admin admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:31.081315 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f1a67-628e-4fd7-8990-e84eab299d07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:31.081727 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f1a67-628e-4fd7-8990-e84eab299d07 admin admin] PATH_INFO: `/v3/projects/bc55a1b8894a43bdaeef8a3ae59bd1dc/users/370896120c7d4fc7b0bf9548336f9382/roles/053e12af84e943e699460a9d76c2072b` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:31.092055 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 790/1599] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:38:31 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:38:31.093016 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc331608-766f-4e13-951b-7def57e1eebd admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:31.097168 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18f1a67-628e-4fd7-8990-e84eab299d07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc55a1b8894a43bdaeef8a3ae59bd1dc, user_id=370896120c7d4fc7b0bf9548336f9382, role_id=053e12af84e943e699460a9d76c2072b)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:31.100534 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18f1a67-628e-4fd7-8990-e84eab299d07 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:31.103819 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=89,keystone:INSERT=11 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:38:31.103963 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc331608-766f-4e13-951b-7def57e1eebd admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:31.108352 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949ccdd2-1167-43b4-a98c-cf829ad8a393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:31.108940 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949ccdd2-1167-43b4-a98c-cf829ad8a393 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:31.109221 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949ccdd2-1167-43b4-a98c-cf829ad8a393 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:31.109496 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949ccdd2-1167-43b4-a98c-cf829ad8a393 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:31.110157 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949ccdd2-1167-43b4-a98c-cf829ad8a393 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:31.111709 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949ccdd2-1167-43b4-a98c-cf829ad8a393 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:31.122122 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 791/1600] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:38:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:38:31.123344 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e9af6d-f486-49d5-86fe-cec7d0007c74 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:31.123634 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e9af6d-f486-49d5-86fe-cec7d0007c74 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:31.123910 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e9af6d-f486-49d5-86fe-cec7d0007c74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:31.152773 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-84e9af6d-f486-49d5-86fe-cec7d0007c74 None None] MFA Rules not processed for user `96bbf934e49b4bff9810a8e11c55b49b`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:38:31.154006 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-84e9af6d-f486-49d5-86fe-cec7d0007c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:31.166462 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:38:31.205369 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 810/1601] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:38:31 2022] PUT /identity/v3/projects/bc55a1b8894a43bdaeef8a3ae59bd1dc/users/370896120c7d4fc7b0bf9548336f9382/roles/053e12af84e943e699460a9d76c2072b => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:38:31.206285 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18f1a67-628e-4fd7-8990-e84eab299d07 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:31.225784 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 792/1602] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:38:31 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:38:31.226525 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e9af6d-f486-49d5-86fe-cec7d0007c74 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:31.237484 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e9af6d-f486-49d5-86fe-cec7d0007c74 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:31.277729 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18f1a67-628e-4fd7-8990-e84eab299d07 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:31.282461 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843ffc08-7e3c-4c9c-ae1d-c58018ba14f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:31.283119 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843ffc08-7e3c-4c9c-ae1d-c58018ba14f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:31.283409 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843ffc08-7e3c-4c9c-ae1d-c58018ba14f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:31.283688 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843ffc08-7e3c-4c9c-ae1d-c58018ba14f7 admin admin] PATH_INFO: `/v3/projects/14017d5edbbd4068b529d5d92444182a/users/5e0c8d7aa4db455a94631becf212866f/roles/053e12af84e943e699460a9d76c2072b` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:31.296370 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b075fe03-902d-4c06-ac98-ef71f4442928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:31.297040 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b075fe03-902d-4c06-ac98-ef71f4442928 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:31.297320 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b075fe03-902d-4c06-ac98-ef71f4442928 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:31.297613 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b075fe03-902d-4c06-ac98-ef71f4442928 admin admin] PATH_INFO: `/v3/projects/9836d4c25e0e46a8a6245bb7b74c55ce/users/1d51e6d2afe1436e8a65b895c83bdd38/roles/053e12af84e943e699460a9d76c2072b` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:31.299276 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843ffc08-7e3c-4c9c-ae1d-c58018ba14f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14017d5edbbd4068b529d5d92444182a, user_id=5e0c8d7aa4db455a94631becf212866f, role_id=053e12af84e943e699460a9d76c2072b)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:31.301977 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843ffc08-7e3c-4c9c-ae1d-c58018ba14f7 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:31.312058 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b075fe03-902d-4c06-ac98-ef71f4442928 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9836d4c25e0e46a8a6245bb7b74c55ce, user_id=1d51e6d2afe1436e8a65b895c83bdd38, role_id=053e12af84e943e699460a9d76c2072b)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:31.316240 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b075fe03-902d-4c06-ac98-ef71f4442928 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:31.399395 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 811/1603] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:38:31 2022] PUT /identity/v3/projects/14017d5edbbd4068b529d5d92444182a/users/5e0c8d7aa4db455a94631becf212866f/roles/053e12af84e943e699460a9d76c2072b => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:38:31.400273 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843ffc08-7e3c-4c9c-ae1d-c58018ba14f7 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:31.427329 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 793/1604] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:38:31 2022] PUT /identity/v3/projects/9836d4c25e0e46a8a6245bb7b74c55ce/users/1d51e6d2afe1436e8a65b895c83bdd38/roles/053e12af84e943e699460a9d76c2072b => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:38:31.428325 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b075fe03-902d-4c06-ac98-ef71f4442928 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:31.467325 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843ffc08-7e3c-4c9c-ae1d-c58018ba14f7 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:31.491997 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b075fe03-902d-4c06-ac98-ef71f4442928 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:31.496607 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8069474-e108-4257-aa7b-0e2b12631750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:31.497269 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8069474-e108-4257-aa7b-0e2b12631750 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:31.497549 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8069474-e108-4257-aa7b-0e2b12631750 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:31.497873 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8069474-e108-4257-aa7b-0e2b12631750 admin admin] PATH_INFO: `/v3/projects/3dfbf0bb45d9413296e05adb32a63fb0/users/5120d3da839c487e917b8bff45044f6e/roles/053e12af84e943e699460a9d76c2072b` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:31.513335 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8069474-e108-4257-aa7b-0e2b12631750 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3dfbf0bb45d9413296e05adb32a63fb0, user_id=5120d3da839c487e917b8bff45044f6e, role_id=053e12af84e943e699460a9d76c2072b)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:31.516034 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8069474-e108-4257-aa7b-0e2b12631750 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:31.524276 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437f8932-60a1-498e-8ec7-4540397bf84b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:31.524937 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437f8932-60a1-498e-8ec7-4540397bf84b admin admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:31.525214 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437f8932-60a1-498e-8ec7-4540397bf84b admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:31.525484 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437f8932-60a1-498e-8ec7-4540397bf84b admin admin] PATH_INFO: `/v3/projects/c59b01df6dd049918fc0fa0cbdedd89f/users/3549490d84fe4650a39e3859fc919a72/roles/6bbb643e0d8d4bc0a9b27c26edf59216` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:31.541033 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437f8932-60a1-498e-8ec7-4540397bf84b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c59b01df6dd049918fc0fa0cbdedd89f, user_id=3549490d84fe4650a39e3859fc919a72, role_id=6bbb643e0d8d4bc0a9b27c26edf59216)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:31.543699 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437f8932-60a1-498e-8ec7-4540397bf84b admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:31.617882 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 794/1605] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:38:31 2022] PUT /identity/v3/projects/3dfbf0bb45d9413296e05adb32a63fb0/users/5120d3da839c487e917b8bff45044f6e/roles/053e12af84e943e699460a9d76c2072b => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:38:31.618912 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8069474-e108-4257-aa7b-0e2b12631750 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:31.635823 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 812/1606] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:38:31 2022] PUT /identity/v3/projects/c59b01df6dd049918fc0fa0cbdedd89f/users/3549490d84fe4650a39e3859fc919a72/roles/6bbb643e0d8d4bc0a9b27c26edf59216 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:38:31.641062 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437f8932-60a1-498e-8ec7-4540397bf84b admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:31.641193 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:38:31.647806 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:38:31.682661 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8069474-e108-4257-aa7b-0e2b12631750 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:31.703544 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437f8932-60a1-498e-8ec7-4540397bf84b admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:31.711233 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372b4ec8-d07f-4dcc-86ea-8a83f89aa4f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:31.712275 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372b4ec8-d07f-4dcc-86ea-8a83f89aa4f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:31.713068 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372b4ec8-d07f-4dcc-86ea-8a83f89aa4f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:31.713551 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372b4ec8-d07f-4dcc-86ea-8a83f89aa4f4 admin admin] PATH_INFO: `/v3/projects/b4a8f2664964492e9c80a595ed043c71/users/56c2f854cf074b1ba2571766b7f2fc51/roles/053e12af84e943e699460a9d76c2072b` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:31.734160 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372b4ec8-d07f-4dcc-86ea-8a83f89aa4f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4a8f2664964492e9c80a595ed043c71, user_id=56c2f854cf074b1ba2571766b7f2fc51, role_id=053e12af84e943e699460a9d76c2072b)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:31.738720 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372b4ec8-d07f-4dcc-86ea-8a83f89aa4f4 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:31.743536 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d14220-ac19-4d78-9286-939f96d359e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:31.746509 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d14220-ac19-4d78-9286-939f96d359e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:31.746509 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d14220-ac19-4d78-9286-939f96d359e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:31.746509 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d14220-ac19-4d78-9286-939f96d359e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:31.746509 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d14220-ac19-4d78-9286-939f96d359e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:31.747215 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d14220-ac19-4d78-9286-939f96d359e8 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:31.757932 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 795/1607] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:38:31 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:38:31.758946 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd670ca7-5f19-4155-aa47-6884d97ad45b None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:31.759234 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd670ca7-5f19-4155-aa47-6884d97ad45b None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:31.759505 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd670ca7-5f19-4155-aa47-6884d97ad45b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:31.787556 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-fd670ca7-5f19-4155-aa47-6884d97ad45b None None] MFA Rules not processed for user `370896120c7d4fc7b0bf9548336f9382`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:38:31.788769 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-fd670ca7-5f19-4155-aa47-6884d97ad45b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:31.823220 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 813/1608] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:38:31 2022] PUT /identity/v3/projects/b4a8f2664964492e9c80a595ed043c71/users/56c2f854cf074b1ba2571766b7f2fc51/roles/053e12af84e943e699460a9d76c2072b => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:38:31.824016 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372b4ec8-d07f-4dcc-86ea-8a83f89aa4f4 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:31.859775 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 796/1609] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:38:31 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:38:31.864804 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5388a60f-16b8-4d3a-8ef2-7b09aa495091 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:31.864804 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5388a60f-16b8-4d3a-8ef2-7b09aa495091 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:31.864804 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5388a60f-16b8-4d3a-8ef2-7b09aa495091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:31.888772 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-5388a60f-16b8-4d3a-8ef2-7b09aa495091 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:38:31.890256 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-5388a60f-16b8-4d3a-8ef2-7b09aa495091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:31.894763 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372b4ec8-d07f-4dcc-86ea-8a83f89aa4f4 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:31.899314 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 797/1610] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:38:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:38:31.900335 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d273d4-329a-4717-ac5b-f17be505bf2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:31.900618 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9fc1c3-727a-45db-82ac-7a0a0b8d2ca8 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:31.900911 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9fc1c3-727a-45db-82ac-7a0a0b8d2ca8 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:31.901186 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9fc1c3-727a-45db-82ac-7a0a0b8d2ca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:31.901289 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d273d4-329a-4717-ac5b-f17be505bf2f admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:31.901594 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d273d4-329a-4717-ac5b-f17be505bf2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:31.904737 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d273d4-329a-4717-ac5b-f17be505bf2f admin admin] PATH_INFO: `/v3/projects` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:31.904737 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d273d4-329a-4717-ac5b-f17be505bf2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:31.904737 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d273d4-329a-4717-ac5b-f17be505bf2f admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:31.921307 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-7f9fc1c3-727a-45db-82ac-7a0a0b8d2ca8 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:38:31.922807 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-7f9fc1c3-727a-45db-82ac-7a0a0b8d2ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:31.931141 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 798/1611] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:38:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:38:31.933418 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f24f740-991b-46c2-9608-454e8203ce59 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:31.933418 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f24f740-991b-46c2-9608-454e8203ce59 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:31.933418 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f24f740-991b-46c2-9608-454e8203ce59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:31.961662 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-2f24f740-991b-46c2-9608-454e8203ce59 None None] MFA Rules not processed for user `5120d3da839c487e917b8bff45044f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:38:31.967761 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-2f24f740-991b-46c2-9608-454e8203ce59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:31.992862 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:38:32.022851 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 814/1612] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:38:31 2022] POST /identity/v3/projects => generated 379 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:38:32.024026 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d273d4-329a-4717-ac5b-f17be505bf2f admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:32.029072 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 799/1613] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:38:31 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:38:32.030090 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f24f740-991b-46c2-9608-454e8203ce59 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:32.043544 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f24f740-991b-46c2-9608-454e8203ce59 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:32.097232 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d273d4-329a-4717-ac5b-f17be505bf2f admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:32.102703 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c044d667-ce0b-49f8-a6e2-37044ad2afdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:32.103449 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c044d667-ce0b-49f8-a6e2-37044ad2afdf admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:32.103730 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c044d667-ce0b-49f8-a6e2-37044ad2afdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:32.103998 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c044d667-ce0b-49f8-a6e2-37044ad2afdf admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:32.104637 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c044d667-ce0b-49f8-a6e2-37044ad2afdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:32.106382 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c044d667-ce0b-49f8-a6e2-37044ad2afdf admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:32.113989 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b47785-849f-4e0c-9523-e116bf7ee091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:32.114646 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b47785-849f-4e0c-9523-e116bf7ee091 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:32.116107 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b47785-849f-4e0c-9523-e116bf7ee091 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:32.116107 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b47785-849f-4e0c-9523-e116bf7ee091 admin admin] PATH_INFO: `/v3/projects/b17284c066e7495587e5125630248284/users/30d986c1d16a4c248889e24349301c86/roles/053e12af84e943e699460a9d76c2072b` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:32.118352 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 815/1614] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:38:32 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:38:32.122651 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7651013-520f-4d8a-9392-1770491f156e None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:32.122651 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7651013-520f-4d8a-9392-1770491f156e None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:32.122651 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7651013-520f-4d8a-9392-1770491f156e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:32.135380 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b47785-849f-4e0c-9523-e116bf7ee091 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b17284c066e7495587e5125630248284, user_id=30d986c1d16a4c248889e24349301c86, role_id=053e12af84e943e699460a9d76c2072b)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:32.138759 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b47785-849f-4e0c-9523-e116bf7ee091 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:32.142446 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-e7651013-520f-4d8a-9392-1770491f156e None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:38:32.143642 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-e7651013-520f-4d8a-9392-1770491f156e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:32.151751 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 816/1615] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:38:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:38:32.152952 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dead09-25a3-4355-866c-1dca7d38f3e6 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:32.153221 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dead09-25a3-4355-866c-1dca7d38f3e6 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:32.153488 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dead09-25a3-4355-866c-1dca7d38f3e6 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:32.154756 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 817/1616] 10.222.0.42 () {54 vars in 962 bytes} [Thu Sep 15 07:38:32 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:38:32.155974 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dead09-25a3-4355-866c-1dca7d38f3e6 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:32.164676 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:38:32.209830 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dead09-25a3-4355-866c-1dca7d38f3e6 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:32.213293 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05210646-387c-4937-a71c-c11c8fa5f19e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:32.213915 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05210646-387c-4937-a71c-c11c8fa5f19e service neutron] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:32.214190 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05210646-387c-4937-a71c-c11c8fa5f19e service neutron] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:32.214462 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05210646-387c-4937-a71c-c11c8fa5f19e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:32.223211 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05210646-387c-4937-a71c-c11c8fa5f19e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:32.225190 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05210646-387c-4937-a71c-c11c8fa5f19e service neutron] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:32.228509 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 800/1617] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:38:32 2022] PUT /identity/v3/projects/b17284c066e7495587e5125630248284/users/30d986c1d16a4c248889e24349301c86/roles/053e12af84e943e699460a9d76c2072b => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:38:32.229427 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b47785-849f-4e0c-9523-e116bf7ee091 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:32.235306 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 818/1618] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:38:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:38:32.235642 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05210646-387c-4937-a71c-c11c8fa5f19e service neutron] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:32.249664 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05210646-387c-4937-a71c-c11c8fa5f19e service neutron] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:32.303280 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b47785-849f-4e0c-9523-e116bf7ee091 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:32.307248 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852ee809-e94f-481e-ba94-4393bfda4e87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:32.307850 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852ee809-e94f-481e-ba94-4393bfda4e87 service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:32.308128 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852ee809-e94f-481e-ba94-4393bfda4e87 service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:32.308397 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852ee809-e94f-481e-ba94-4393bfda4e87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:32.312418 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3475024d-4568-4c8d-ae12-30a381393b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:32.313156 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3475024d-4568-4c8d-ae12-30a381393b37 admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:32.313446 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3475024d-4568-4c8d-ae12-30a381393b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:32.313776 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3475024d-4568-4c8d-ae12-30a381393b37 admin admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:32.314443 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3475024d-4568-4c8d-ae12-30a381393b37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:32.316685 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3475024d-4568-4c8d-ae12-30a381393b37 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:32.322666 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852ee809-e94f-481e-ba94-4393bfda4e87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:32.324197 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852ee809-e94f-481e-ba94-4393bfda4e87 service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:32.334648 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 801/1619] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:38:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:38:32.336235 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bf78c7-4c93-458f-a639-961ae9d18517 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:32.336589 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bf78c7-4c93-458f-a639-961ae9d18517 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:32.336972 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bf78c7-4c93-458f-a639-961ae9d18517 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:32.338262 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 802/1620] 10.222.0.42 () {54 vars in 962 bytes} [Thu Sep 15 07:38:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:38:32.344434 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bf78c7-4c93-458f-a639-961ae9d18517 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:32.355917 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bf78c7-4c93-458f-a639-961ae9d18517 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:32.361096 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e6bf84-ad0d-4865-8a39-adb646b8bd8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:32.361817 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e6bf84-ad0d-4865-8a39-adb646b8bd8d admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:32.362100 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e6bf84-ad0d-4865-8a39-adb646b8bd8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:32.362492 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e6bf84-ad0d-4865-8a39-adb646b8bd8d admin admin] PATH_INFO: `/v3/projects/c59b01df6dd049918fc0fa0cbdedd89f/users/3549490d84fe4650a39e3859fc919a72/roles/6bbb643e0d8d4bc0a9b27c26edf59216` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:32.364536 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e6bf84-ad0d-4865-8a39-adb646b8bd8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c59b01df6dd049918fc0fa0cbdedd89f, user_id=3549490d84fe4650a39e3859fc919a72, role_id=6bbb643e0d8d4bc0a9b27c26edf59216)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:32.367563 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e6bf84-ad0d-4865-8a39-adb646b8bd8d admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:32.378319 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 803/1621] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:38:32 2022] PUT /identity/v3/projects/c59b01df6dd049918fc0fa0cbdedd89f/users/3549490d84fe4650a39e3859fc919a72/roles/6bbb643e0d8d4bc0a9b27c26edf59216 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:38:32.379450 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e6bf84-ad0d-4865-8a39-adb646b8bd8d admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:32.395330 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 819/1622] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:38:32 2022] POST /identity/v3/users => generated 296 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:38:32.397299 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cd5fb4-636f-407a-ad18-853add952247 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:32.397299 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cd5fb4-636f-407a-ad18-853add952247 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:32.397299 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cd5fb4-636f-407a-ad18-853add952247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:32.444378 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:38:32.455035 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e6bf84-ad0d-4865-8a39-adb646b8bd8d admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:32.458869 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5952e6-a8dd-491e-bdd9-6d9061399cf6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:32.459635 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5952e6-a8dd-491e-bdd9-6d9061399cf6 service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:32.459968 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5952e6-a8dd-491e-bdd9-6d9061399cf6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:32.460196 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5952e6-a8dd-491e-bdd9-6d9061399cf6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:32.471019 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-a1cd5fb4-636f-407a-ad18-853add952247 None None] MFA Rules not processed for user `a080c7ab9b684abca6828067f2572c3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:38:32.472222 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-a1cd5fb4-636f-407a-ad18-853add952247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:32.527269 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5952e6-a8dd-491e-bdd9-6d9061399cf6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:32.528741 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5952e6-a8dd-491e-bdd9-6d9061399cf6 service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:32.539195 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 804/1623] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:38:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:38:32.540502 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb368f-cb0f-4582-933e-3f3aeefc0060 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:32.540797 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb368f-cb0f-4582-933e-3f3aeefc0060 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:32.541188 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb368f-cb0f-4582-933e-3f3aeefc0060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:32.557571 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 820/1624] 10.222.0.42 () {58 vars in 1059 bytes} [Thu Sep 15 07:38:32 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:38:32.559673 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd57fdd7-e9e9-46e1-96fc-2effb233af1d None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:32.560081 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd57fdd7-e9e9-46e1-96fc-2effb233af1d None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:32.560789 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd57fdd7-e9e9-46e1-96fc-2effb233af1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:32.578638 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-0fbb368f-cb0f-4582-933e-3f3aeefc0060 None None] MFA Rules not processed for user `30d986c1d16a4c248889e24349301c86`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:38:32.580148 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-0fbb368f-cb0f-4582-933e-3f3aeefc0060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:32.586643 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-dd57fdd7-e9e9-46e1-96fc-2effb233af1d None None] MFA Rules not processed for user `a080c7ab9b684abca6828067f2572c3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:38:32.586643 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-dd57fdd7-e9e9-46e1-96fc-2effb233af1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:32.603711 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 821/1625] 10.222.0.42 () {58 vars in 1059 bytes} [Thu Sep 15 07:38:32 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:38:32.605295 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a603c4-624b-44f3-843f-4ead20a71ca1 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:32.605754 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a603c4-624b-44f3-843f-4ead20a71ca1 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:32.606082 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a603c4-624b-44f3-843f-4ead20a71ca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:32.628424 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:38:32.637854 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-91a603c4-624b-44f3-843f-4ead20a71ca1 None None] MFA Rules not processed for user `3549490d84fe4650a39e3859fc919a72`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:38:32.644089 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-91a603c4-624b-44f3-843f-4ead20a71ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:32.656193 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 805/1626] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:38:32 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:38:32.663719 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbb368f-cb0f-4582-933e-3f3aeefc0060 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:32.683634 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbb368f-cb0f-4582-933e-3f3aeefc0060 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:32.686764 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda5d4f1-0356-43a2-b46b-14497d604d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:32.687595 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda5d4f1-0356-43a2-b46b-14497d604d42 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:32.688025 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda5d4f1-0356-43a2-b46b-14497d604d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:32.688950 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda5d4f1-0356-43a2-b46b-14497d604d42 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:32.689527 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda5d4f1-0356-43a2-b46b-14497d604d42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:32.694224 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda5d4f1-0356-43a2-b46b-14497d604d42 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:32.722264 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 806/1627] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:38:32 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:38:32.723927 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda5d4f1-0356-43a2-b46b-14497d604d42 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:32.741116 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 822/1628] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:38:32 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:38:32.742398 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a603c4-624b-44f3-843f-4ead20a71ca1 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:32.766533 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a603c4-624b-44f3-843f-4ead20a71ca1 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:32.774762 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456a299c-e02e-4a55-b87e-232d6a69af81 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a080c7ab9b684abca6828067f2572c3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:32.776100 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456a299c-e02e-4a55-b87e-232d6a69af81 service nova] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:32.776481 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456a299c-e02e-4a55-b87e-232d6a69af81 service nova] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:32.777056 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456a299c-e02e-4a55-b87e-232d6a69af81 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:32.778185 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda5d4f1-0356-43a2-b46b-14497d604d42 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:32.793336 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba341620-5a0a-43e9-853b-e575e6b1de8f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a080c7ab9b684abca6828067f2572c3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:32.794469 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba341620-5a0a-43e9-853b-e575e6b1de8f service nova] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:32.794762 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba341620-5a0a-43e9-853b-e575e6b1de8f service nova] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:32.795246 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba341620-5a0a-43e9-853b-e575e6b1de8f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:32.858190 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456a299c-e02e-4a55-b87e-232d6a69af81 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:32.862234 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456a299c-e02e-4a55-b87e-232d6a69af81 service nova] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:32.863817 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba341620-5a0a-43e9-853b-e575e6b1de8f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:32.865616 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba341620-5a0a-43e9-853b-e575e6b1de8f service nova] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:32.876889 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 807/1629] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:38:32 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:38:32.877766 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba341620-5a0a-43e9-853b-e575e6b1de8f service nova] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:32.889841 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba341620-5a0a-43e9-853b-e575e6b1de8f service nova] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:32.894997 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 823/1630] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:38:32 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:38:32.895271 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67453684-1fc8-48be-a957-732307b7aa2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:32.895919 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67453684-1fc8-48be-a957-732307b7aa2b admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:32.896712 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67453684-1fc8-48be-a957-732307b7aa2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:32.897296 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67453684-1fc8-48be-a957-732307b7aa2b admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:32.897412 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456a299c-e02e-4a55-b87e-232d6a69af81 service nova] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:32.898067 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67453684-1fc8-48be-a957-732307b7aa2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:32.900217 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67453684-1fc8-48be-a957-732307b7aa2b admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:32.908575 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:38:32.920511 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456a299c-e02e-4a55-b87e-232d6a69af81 service nova] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:32.931345 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f168b746-4e77-4969-9491-fedb5eb5b0f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:32.932035 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f168b746-4e77-4969-9491-fedb5eb5b0f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:32.932317 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f168b746-4e77-4969-9491-fedb5eb5b0f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:32.932594 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f168b746-4e77-4969-9491-fedb5eb5b0f7 admin admin] PATH_INFO: `/v3/projects/b6a4ddb789f04146b628a5ef7e5d4d4d/users/1e4bddfa72b44a37adabc00a1099944f/roles/6bbb643e0d8d4bc0a9b27c26edf59216` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:32.986149 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f168b746-4e77-4969-9491-fedb5eb5b0f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6a4ddb789f04146b628a5ef7e5d4d4d, user_id=1e4bddfa72b44a37adabc00a1099944f, role_id=6bbb643e0d8d4bc0a9b27c26edf59216)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:32.988773 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f168b746-4e77-4969-9491-fedb5eb5b0f7 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:33.037852 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 808/1631] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:38:32 2022] POST /identity/v3/projects => generated 405 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:38:33.037852 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67453684-1fc8-48be-a957-732307b7aa2b admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:33.116542 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 824/1632] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:38:32 2022] PUT /identity/v3/projects/b6a4ddb789f04146b628a5ef7e5d4d4d/users/1e4bddfa72b44a37adabc00a1099944f/roles/6bbb643e0d8d4bc0a9b27c26edf59216 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:38:33.117502 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f168b746-4e77-4969-9491-fedb5eb5b0f7 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:33.140555 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67453684-1fc8-48be-a957-732307b7aa2b admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:33.259761 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f168b746-4e77-4969-9491-fedb5eb5b0f7 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:33.265406 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4acae3-b12a-46f3-9f9a-876720a95043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:33.266101 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02967b1c-155d-4559-b274-ad888f5ab06b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecf6642b7b84d3c9756bb41ab3349d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:33.267073 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02967b1c-155d-4559-b274-ad888f5ab06b service cinder] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:33.267400 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4acae3-b12a-46f3-9f9a-876720a95043 admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:33.267861 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4acae3-b12a-46f3-9f9a-876720a95043 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:33.268311 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4acae3-b12a-46f3-9f9a-876720a95043 admin admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:33.268658 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02967b1c-155d-4559-b274-ad888f5ab06b service cinder] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:33.269432 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4acae3-b12a-46f3-9f9a-876720a95043 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:33.270950 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02967b1c-155d-4559-b274-ad888f5ab06b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:33.272551 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4acae3-b12a-46f3-9f9a-876720a95043 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:33.355944 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:38:33.362245 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 825/1633] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:38:33 2022] POST /identity/v3/users => generated 309 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:38:33.365687 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4acae3-b12a-46f3-9f9a-876720a95043 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:33.375680 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02967b1c-155d-4559-b274-ad888f5ab06b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:33.376225 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4acae3-b12a-46f3-9f9a-876720a95043 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:33.377887 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02967b1c-155d-4559-b274-ad888f5ab06b service cinder] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:33.381924 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4069338-2f69-4940-9ef0-f9c9b01a511f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:33.382576 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4069338-2f69-4940-9ef0-f9c9b01a511f admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:33.382863 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4069338-2f69-4940-9ef0-f9c9b01a511f admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:33.383141 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4069338-2f69-4940-9ef0-f9c9b01a511f admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:33.383799 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4069338-2f69-4940-9ef0-f9c9b01a511f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:33.385421 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4069338-2f69-4940-9ef0-f9c9b01a511f admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:33.393329 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 809/1634] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:38:33 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:38:33.395069 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37a3e77-aa40-4846-a713-0d4add8c765f None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:33.395534 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37a3e77-aa40-4846-a713-0d4add8c765f None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:33.395975 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37a3e77-aa40-4846-a713-0d4add8c765f None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:33.398160 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 810/1635] 10.222.0.42 () {54 vars in 912 bytes} [Thu Sep 15 07:38:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:38:33.399745 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0d7c57-4434-42d4-adae-b12bd5d88591 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:33.400182 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0d7c57-4434-42d4-adae-b12bd5d88591 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:33.400605 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0d7c57-4434-42d4-adae-b12bd5d88591 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:33.406593 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 811/1636] 10.222.0.42 () {54 vars in 912 bytes} [Thu Sep 15 07:38:33 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:38:33.408275 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0d7c57-4434-42d4-adae-b12bd5d88591 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:33.412762 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 826/1637] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:38:33 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:38:33.421983 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06396627-e5e0-4aee-a2e3-8318778a26ae None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:33.421983 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06396627-e5e0-4aee-a2e3-8318778a26ae None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:33.421983 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06396627-e5e0-4aee-a2e3-8318778a26ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:33.426412 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0d7c57-4434-42d4-adae-b12bd5d88591 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:33.434067 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7062467-6b84-4342-9101-c0e035882512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:33.434949 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7062467-6b84-4342-9101-c0e035882512 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:33.435699 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7062467-6b84-4342-9101-c0e035882512 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:33.436139 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7062467-6b84-4342-9101-c0e035882512 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:33.436936 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7062467-6b84-4342-9101-c0e035882512 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:33.439087 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7062467-6b84-4342-9101-c0e035882512 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:33.452399 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 812/1638] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:38:33 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:38:33.453718 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a8256d-670f-41b0-8a82-16c0300105d3 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:33.454105 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a8256d-670f-41b0-8a82-16c0300105d3 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:33.454499 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a8256d-670f-41b0-8a82-16c0300105d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:33.456172 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-06396627-e5e0-4aee-a2e3-8318778a26ae None None] MFA Rules not processed for user `a080c7ab9b684abca6828067f2572c3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:38:33.457419 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-06396627-e5e0-4aee-a2e3-8318778a26ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:33.483546 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-25a8256d-670f-41b0-8a82-16c0300105d3 None None] MFA Rules not processed for user `a080c7ab9b684abca6828067f2572c3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:38:33.486821 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-25a8256d-670f-41b0-8a82-16c0300105d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:33.556484 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:38:33.610258 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 827/1639] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Sep 15 07:38:33 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:38:33.611581 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06396627-e5e0-4aee-a2e3-8318778a26ae None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:33.628078 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06396627-e5e0-4aee-a2e3-8318778a26ae None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:33.632305 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 813/1640] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Sep 15 07:38:33 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:38:33.633709 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a8256d-670f-41b0-8a82-16c0300105d3 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:33.636876 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cadc995-05af-4402-b089-06921b119894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:33.636876 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cadc995-05af-4402-b089-06921b119894 admin admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:33.637088 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cadc995-05af-4402-b089-06921b119894 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:33.637368 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cadc995-05af-4402-b089-06921b119894 admin admin] PATH_INFO: `/v3/projects/b6a4ddb789f04146b628a5ef7e5d4d4d/users/1e4bddfa72b44a37adabc00a1099944f/roles/6bbb643e0d8d4bc0a9b27c26edf59216` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:33.643820 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cadc995-05af-4402-b089-06921b119894 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6a4ddb789f04146b628a5ef7e5d4d4d, user_id=1e4bddfa72b44a37adabc00a1099944f, role_id=6bbb643e0d8d4bc0a9b27c26edf59216)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:33.647787 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cadc995-05af-4402-b089-06921b119894 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:33.650715 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a8256d-670f-41b0-8a82-16c0300105d3 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:33.659802 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e73365e-a688-45ce-896c-b52944b35ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:33.660602 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e73365e-a688-45ce-896c-b52944b35ef0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:33.661473 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e73365e-a688-45ce-896c-b52944b35ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:33.662118 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e73365e-a688-45ce-896c-b52944b35ef0 admin admin] PATH_INFO: `/v3/projects/a22520811f1845d797ad8721a0e8f4e8/users/831b4c254a574c45a760c68be61bd2ef/roles/6bbb643e0d8d4bc0a9b27c26edf59216` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:33.674846 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 828/1641] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:38:33 2022] PUT /identity/v3/projects/b6a4ddb789f04146b628a5ef7e5d4d4d/users/1e4bddfa72b44a37adabc00a1099944f/roles/6bbb643e0d8d4bc0a9b27c26edf59216 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:38:33.678255 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cadc995-05af-4402-b089-06921b119894 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:33.690650 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e73365e-a688-45ce-896c-b52944b35ef0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a22520811f1845d797ad8721a0e8f4e8, user_id=831b4c254a574c45a760c68be61bd2ef, role_id=6bbb643e0d8d4bc0a9b27c26edf59216)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:33.694239 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e73365e-a688-45ce-896c-b52944b35ef0 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:33.752695 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cadc995-05af-4402-b089-06921b119894 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:33.756798 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5015db-db70-44be-81b9-42bccfa1adb2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecf6642b7b84d3c9756bb41ab3349d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:33.757404 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5015db-db70-44be-81b9-42bccfa1adb2 service cinder] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:33.757726 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5015db-db70-44be-81b9-42bccfa1adb2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:33.757996 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5015db-db70-44be-81b9-42bccfa1adb2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:33.816777 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 814/1642] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:38:33 2022] PUT /identity/v3/projects/a22520811f1845d797ad8721a0e8f4e8/users/831b4c254a574c45a760c68be61bd2ef/roles/6bbb643e0d8d4bc0a9b27c26edf59216 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:38:33.816777 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93c55f5-6a8c-4efe-af1e-a213cb69bf5a None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:33.816777 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93c55f5-6a8c-4efe-af1e-a213cb69bf5a None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:33.816777 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93c55f5-6a8c-4efe-af1e-a213cb69bf5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:33.831438 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5015db-db70-44be-81b9-42bccfa1adb2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:33.832903 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5015db-db70-44be-81b9-42bccfa1adb2 service cinder] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:33.844173 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 829/1643] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:38:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:38:33.847388 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5015db-db70-44be-81b9-42bccfa1adb2 service cinder] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:33.847593 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-c93c55f5-6a8c-4efe-af1e-a213cb69bf5a None None] MFA Rules not processed for user `1e4bddfa72b44a37adabc00a1099944f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:38:33.848723 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-c93c55f5-6a8c-4efe-af1e-a213cb69bf5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:33.853743 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5015db-db70-44be-81b9-42bccfa1adb2 service cinder] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:33.909654 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:38:33.933073 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc17ec4-62db-4c90-90ae-fff9804c15b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecf6642b7b84d3c9756bb41ab3349d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:33.933866 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc17ec4-62db-4c90-90ae-fff9804c15b4 service cinder] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:33.934333 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc17ec4-62db-4c90-90ae-fff9804c15b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:33.935572 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc17ec4-62db-4c90-90ae-fff9804c15b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:33.955774 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 815/1644] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:38:33 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:38:33.957785 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93c55f5-6a8c-4efe-af1e-a213cb69bf5a None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:34.043491 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc17ec4-62db-4c90-90ae-fff9804c15b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:34.045784 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc17ec4-62db-4c90-90ae-fff9804c15b4 service cinder] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:34.071026 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 830/1645] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:38:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:38:34.110774 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93c55f5-6a8c-4efe-af1e-a213cb69bf5a None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:34.120072 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b56b3de-68a7-4f79-8a54-5f6679077b86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:34.121180 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b56b3de-68a7-4f79-8a54-5f6679077b86 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:34.122478 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b56b3de-68a7-4f79-8a54-5f6679077b86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:34.123015 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b56b3de-68a7-4f79-8a54-5f6679077b86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:34.123845 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b56b3de-68a7-4f79-8a54-5f6679077b86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:34.125985 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b56b3de-68a7-4f79-8a54-5f6679077b86 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:34.155071 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 816/1646] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:38:33 2022] GET /identity/v3/roles => generated 1420 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:38:34.159158 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc17ec4-62db-4c90-90ae-fff9804c15b4 service cinder] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:34.176861 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc17ec4-62db-4c90-90ae-fff9804c15b4 service cinder] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:34.186225 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60430ff-e875-4f47-8608-95ace471e5ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:34.186907 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60430ff-e875-4f47-8608-95ace471e5ae admin admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:34.187187 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60430ff-e875-4f47-8608-95ace471e5ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:34.187462 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60430ff-e875-4f47-8608-95ace471e5ae admin admin] PATH_INFO: `/v3/projects/a22520811f1845d797ad8721a0e8f4e8/users/831b4c254a574c45a760c68be61bd2ef/roles/6bbb643e0d8d4bc0a9b27c26edf59216` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:34.189887 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60430ff-e875-4f47-8608-95ace471e5ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=a22520811f1845d797ad8721a0e8f4e8, user_id=831b4c254a574c45a760c68be61bd2ef, role_id=6bbb643e0d8d4bc0a9b27c26edf59216)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:34.192559 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60430ff-e875-4f47-8608-95ace471e5ae admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:34.227097 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 831/1647] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:38:34 2022] PUT /identity/v3/projects/a22520811f1845d797ad8721a0e8f4e8/users/831b4c254a574c45a760c68be61bd2ef/roles/6bbb643e0d8d4bc0a9b27c26edf59216 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:38:34.229253 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a65a0e4-b2c8-4165-bc7b-374b89d8d619 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:34.229804 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a65a0e4-b2c8-4165-bc7b-374b89d8d619 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:34.230139 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a65a0e4-b2c8-4165-bc7b-374b89d8d619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:34.302272 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-0a65a0e4-b2c8-4165-bc7b-374b89d8d619 None None] MFA Rules not processed for user `831b4c254a574c45a760c68be61bd2ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:38:34.313667 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-0a65a0e4-b2c8-4165-bc7b-374b89d8d619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:34.360245 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:38:34.395972 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 817/1648] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:38:34 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 07:38:35.383446 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60430ff-e875-4f47-8608-95ace471e5ae admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:35.383446 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60430ff-e875-4f47-8608-95ace471e5ae admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:35.383446 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae5b54a-dc77-4d61-ab57-bdb1f54300de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecf6642b7b84d3c9756bb41ab3349d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:35.383446 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae5b54a-dc77-4d61-ab57-bdb1f54300de service cinder] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:35.385422 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae5b54a-dc77-4d61-ab57-bdb1f54300de service cinder] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:35.385422 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae5b54a-dc77-4d61-ab57-bdb1f54300de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:35.385422 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:38:35.397605 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae5b54a-dc77-4d61-ab57-bdb1f54300de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:35.397605 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae5b54a-dc77-4d61-ab57-bdb1f54300de service cinder] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:35.407654 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 832/1649] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:38:35 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:38:35.530670 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a65a0e4-b2c8-4165-bc7b-374b89d8d619 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:35.549646 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a65a0e4-b2c8-4165-bc7b-374b89d8d619 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:35.554825 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002b5c2f-f683-4101-837a-fc8a53ecbf9b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecf6642b7b84d3c9756bb41ab3349d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:35.555475 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002b5c2f-f683-4101-837a-fc8a53ecbf9b service cinder] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:35.555766 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002b5c2f-f683-4101-837a-fc8a53ecbf9b service cinder] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:35.556045 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002b5c2f-f683-4101-837a-fc8a53ecbf9b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:35.566782 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002b5c2f-f683-4101-837a-fc8a53ecbf9b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:35.568442 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002b5c2f-f683-4101-837a-fc8a53ecbf9b service cinder] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:35.579707 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 818/1650] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:38:35 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:38:36.035703 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae5b54a-dc77-4d61-ab57-bdb1f54300de service cinder] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:36.048267 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae5b54a-dc77-4d61-ab57-bdb1f54300de service cinder] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:36.055485 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52664a0c-c97f-4ff4-8795-8da3fd610ce4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecf6642b7b84d3c9756bb41ab3349d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:36.057801 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52664a0c-c97f-4ff4-8795-8da3fd610ce4 service cinder] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:36.057801 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52664a0c-c97f-4ff4-8795-8da3fd610ce4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:36.057801 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52664a0c-c97f-4ff4-8795-8da3fd610ce4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:36.194980 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52664a0c-c97f-4ff4-8795-8da3fd610ce4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:36.196417 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52664a0c-c97f-4ff4-8795-8da3fd610ce4 service cinder] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:36.206437 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002b5c2f-f683-4101-837a-fc8a53ecbf9b service cinder] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:36.218965 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 833/1651] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:38:36 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:38:36.260526 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52664a0c-c97f-4ff4-8795-8da3fd610ce4 service cinder] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:36.333898 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002b5c2f-f683-4101-837a-fc8a53ecbf9b service cinder] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:36.341846 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb5b266-da44-495f-8ec7-e4057d06476d tempest-VolumeQuotaClassesTest-320337428 tempest-VolumeQuotaClassesTest-320337428-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3549490d84fe4650a39e3859fc919a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c59b01df6dd049918fc0fa0cbdedd89f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:36.342987 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb5b266-da44-495f-8ec7-e4057d06476d tempest-VolumeQuotaClassesTest-320337428 tempest-VolumeQuotaClassesTest-320337428-project-admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:36.343171 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb5b266-da44-495f-8ec7-e4057d06476d tempest-VolumeQuotaClassesTest-320337428 tempest-VolumeQuotaClassesTest-320337428-project-admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:36.343549 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb5b266-da44-495f-8ec7-e4057d06476d tempest-VolumeQuotaClassesTest-320337428 tempest-VolumeQuotaClassesTest-320337428-project-admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:36.345171 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb5b266-da44-495f-8ec7-e4057d06476d tempest-VolumeQuotaClassesTest-320337428 tempest-VolumeQuotaClassesTest-320337428-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:38:36.346038 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb5b266-da44-495f-8ec7-e4057d06476d tempest-VolumeQuotaClassesTest-320337428 tempest-VolumeQuotaClassesTest-320337428-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:36.348595 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb5b266-da44-495f-8ec7-e4057d06476d tempest-VolumeQuotaClassesTest-320337428 tempest-VolumeQuotaClassesTest-320337428-project-admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:36.360132 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52664a0c-c97f-4ff4-8795-8da3fd610ce4 service cinder] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:36.367256 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ec7293-8800-4ed1-bd98-75385217a44c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:36.367714 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ec7293-8800-4ed1-bd98-75385217a44c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:36.367998 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ec7293-8800-4ed1-bd98-75385217a44c admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:36.368273 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ec7293-8800-4ed1-bd98-75385217a44c admin admin] PATH_INFO: `/v3/users/30d986c1d16a4c248889e24349301c86` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:36.369453 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ec7293-8800-4ed1-bd98-75385217a44c admin admin] RBAC: Authorizing `identity:delete_user(user_id=30d986c1d16a4c248889e24349301c86)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:36.370814 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: WARNING py.warnings [None req-e7ec7293-8800-4ed1-bd98-75385217a44c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 15 07:38:36.370814 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: warnings.warn(msg) Sep 15 07:38:36.370814 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: Sep 15 07:38:36.372096 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ec7293-8800-4ed1-bd98-75385217a44c admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:36.373280 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 819/1652] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 15 07:38:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:38:36.377861 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb5b266-da44-495f-8ec7-e4057d06476d tempest-VolumeQuotaClassesTest-320337428 tempest-VolumeQuotaClassesTest-320337428-project-admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:36.394036 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb5b266-da44-495f-8ec7-e4057d06476d tempest-VolumeQuotaClassesTest-320337428 tempest-VolumeQuotaClassesTest-320337428-project-admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:36.400316 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7734f55b-b929-4b5d-9ee1-2a755b611701 tempest-VolumeQuotaClassesTest-320337428 tempest-VolumeQuotaClassesTest-320337428-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3549490d84fe4650a39e3859fc919a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c59b01df6dd049918fc0fa0cbdedd89f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:36.401050 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7734f55b-b929-4b5d-9ee1-2a755b611701 tempest-VolumeQuotaClassesTest-320337428 tempest-VolumeQuotaClassesTest-320337428-project-admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:36.401500 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7734f55b-b929-4b5d-9ee1-2a755b611701 tempest-VolumeQuotaClassesTest-320337428 tempest-VolumeQuotaClassesTest-320337428-project-admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:36.401824 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7734f55b-b929-4b5d-9ee1-2a755b611701 tempest-VolumeQuotaClassesTest-320337428 tempest-VolumeQuotaClassesTest-320337428-project-admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:36.402534 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7734f55b-b929-4b5d-9ee1-2a755b611701 tempest-VolumeQuotaClassesTest-320337428 tempest-VolumeQuotaClassesTest-320337428-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:36.404470 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7734f55b-b929-4b5d-9ee1-2a755b611701 tempest-VolumeQuotaClassesTest-320337428 tempest-VolumeQuotaClassesTest-320337428-project-admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:36.518507 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 820/1653] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:38:36 2022] POST /identity/v3/projects => generated 355 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:38:36.520345 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7734f55b-b929-4b5d-9ee1-2a755b611701 tempest-VolumeQuotaClassesTest-320337428 tempest-VolumeQuotaClassesTest-320337428-project-admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:36.626065 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-e7ec7293-8800-4ed1-bd98-75385217a44c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30d986c1d16a4c248889e24349301c86'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:38:36.659775 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7734f55b-b929-4b5d-9ee1-2a755b611701 tempest-VolumeQuotaClassesTest-320337428 tempest-VolumeQuotaClassesTest-320337428-project-admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:36.667078 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65468619-634e-4fdc-a404-4d1103f7957a tempest-VolumeQuotaClassesTest-320337428 tempest-VolumeQuotaClassesTest-320337428-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3549490d84fe4650a39e3859fc919a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c59b01df6dd049918fc0fa0cbdedd89f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:36.668495 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65468619-634e-4fdc-a404-4d1103f7957a tempest-VolumeQuotaClassesTest-320337428 tempest-VolumeQuotaClassesTest-320337428-project-admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:36.668790 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65468619-634e-4fdc-a404-4d1103f7957a tempest-VolumeQuotaClassesTest-320337428 tempest-VolumeQuotaClassesTest-320337428-project-admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:36.669067 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65468619-634e-4fdc-a404-4d1103f7957a tempest-VolumeQuotaClassesTest-320337428 tempest-VolumeQuotaClassesTest-320337428-project-admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:36.669846 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65468619-634e-4fdc-a404-4d1103f7957a tempest-VolumeQuotaClassesTest-320337428 tempest-VolumeQuotaClassesTest-320337428-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:38:36.670143 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65468619-634e-4fdc-a404-4d1103f7957a tempest-VolumeQuotaClassesTest-320337428 tempest-VolumeQuotaClassesTest-320337428-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:36.672161 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65468619-634e-4fdc-a404-4d1103f7957a tempest-VolumeQuotaClassesTest-320337428 tempest-VolumeQuotaClassesTest-320337428-project-admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:36.706150 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 821/1654] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 15 07:38:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:38:36.727646 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-e7ec7293-8800-4ed1-bd98-75385217a44c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30d986c1d16a4c248889e24349301c86'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:38:36.728763 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-e7ec7293-8800-4ed1-bd98-75385217a44c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30d986c1d16a4c248889e24349301c86'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:38:36.729520 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-e7ec7293-8800-4ed1-bd98-75385217a44c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30d986c1d16a4c248889e24349301c86'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:38:36.750765 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65468619-634e-4fdc-a404-4d1103f7957a tempest-VolumeQuotaClassesTest-320337428 tempest-VolumeQuotaClassesTest-320337428-project-admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:36.753102 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-65468619-634e-4fdc-a404-4d1103f7957a tempest-VolumeQuotaClassesTest-320337428 tempest-VolumeQuotaClassesTest-320337428-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:36.770181 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65468619-634e-4fdc-a404-4d1103f7957a tempest-VolumeQuotaClassesTest-320337428 tempest-VolumeQuotaClassesTest-320337428-project-admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:36.770927 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-e7ec7293-8800-4ed1-bd98-75385217a44c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30d986c1d16a4c248889e24349301c86'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:38:36.776218 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c523564-8c19-43e4-8983-56c7c957d225 tempest-VolumeQuotaClassesTest-320337428 tempest-VolumeQuotaClassesTest-320337428-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3549490d84fe4650a39e3859fc919a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c59b01df6dd049918fc0fa0cbdedd89f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:36.777033 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c523564-8c19-43e4-8983-56c7c957d225 tempest-VolumeQuotaClassesTest-320337428 tempest-VolumeQuotaClassesTest-320337428-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:36.777459 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c523564-8c19-43e4-8983-56c7c957d225 tempest-VolumeQuotaClassesTest-320337428 tempest-VolumeQuotaClassesTest-320337428-project-admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:36.777908 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c523564-8c19-43e4-8983-56c7c957d225 tempest-VolumeQuotaClassesTest-320337428 tempest-VolumeQuotaClassesTest-320337428-project-admin] PATH_INFO: `/v3/projects/53cd27bea0da4cacab5cf19860d49c5f` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:36.779510 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c523564-8c19-43e4-8983-56c7c957d225 tempest-VolumeQuotaClassesTest-320337428 tempest-VolumeQuotaClassesTest-320337428-project-admin] RBAC: Authorizing `identity:delete_project(project_id=53cd27bea0da4cacab5cf19860d49c5f)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:36.780765 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: WARNING py.warnings [None req-2c523564-8c19-43e4-8983-56c7c957d225 tempest-VolumeQuotaClassesTest-320337428 tempest-VolumeQuotaClassesTest-320337428-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 15 07:38:36.780765 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: warnings.warn(msg) Sep 15 07:38:36.780765 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: Sep 15 07:38:36.782047 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c523564-8c19-43e4-8983-56c7c957d225 tempest-VolumeQuotaClassesTest-320337428 tempest-VolumeQuotaClassesTest-320337428-project-admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:36.785504 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:38:36.851392 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 834/1655] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:38:36 2022] DELETE /identity/v3/users/30d986c1d16a4c248889e24349301c86 => generated 0 bytes in 586 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 07:38:36.854053 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654a1f92-c228-4379-aace-f29f66d04ea0 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:36.857035 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654a1f92-c228-4379-aace-f29f66d04ea0 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:36.858040 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654a1f92-c228-4379-aace-f29f66d04ea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:36.878614 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:38:36.892688 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-654a1f92-c228-4379-aace-f29f66d04ea0 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:38:36.894008 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-654a1f92-c228-4379-aace-f29f66d04ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:36.992688 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-2c523564-8c19-43e4-8983-56c7c957d225 tempest-VolumeQuotaClassesTest-320337428 tempest-VolumeQuotaClassesTest-320337428-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53cd27bea0da4cacab5cf19860d49c5f', 'request_id': 'req-2c523564-8c19-43e4-8983-56c7c957d225'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:38:37.015422 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-2c523564-8c19-43e4-8983-56c7c957d225 tempest-VolumeQuotaClassesTest-320337428 tempest-VolumeQuotaClassesTest-320337428-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53cd27bea0da4cacab5cf19860d49c5f', 'request_id': 'req-2c523564-8c19-43e4-8983-56c7c957d225'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:38:37.097635 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 835/1656] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:38:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:38:37.115048 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-2c523564-8c19-43e4-8983-56c7c957d225 tempest-VolumeQuotaClassesTest-320337428 tempest-VolumeQuotaClassesTest-320337428-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53cd27bea0da4cacab5cf19860d49c5f', 'request_id': 'req-2c523564-8c19-43e4-8983-56c7c957d225'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:38:37.161718 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-2c523564-8c19-43e4-8983-56c7c957d225 tempest-VolumeQuotaClassesTest-320337428 tempest-VolumeQuotaClassesTest-320337428-project-admin] The token cache is being invalidate because project 53cd27bea0da4cacab5cf19860d49c5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:38:37.161718 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-2c523564-8c19-43e4-8983-56c7c957d225 tempest-VolumeQuotaClassesTest-320337428 tempest-VolumeQuotaClassesTest-320337428-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:38:37.161718 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-2c523564-8c19-43e4-8983-56c7c957d225 tempest-VolumeQuotaClassesTest-320337428 tempest-VolumeQuotaClassesTest-320337428-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:38:37.161718 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 822/1657] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:38:36 2022] DELETE /identity/v3/projects/53cd27bea0da4cacab5cf19860d49c5f => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:38:37.193681 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654a1f92-c228-4379-aace-f29f66d04ea0 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:37.195481 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-654a1f92-c228-4379-aace-f29f66d04ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:37.313235 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654a1f92-c228-4379-aace-f29f66d04ea0 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:37.406135 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae74d5e-0918-456b-be5d-dfca93822b85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:37.406974 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae74d5e-0918-456b-be5d-dfca93822b85 service neutron] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:37.407285 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae74d5e-0918-456b-be5d-dfca93822b85 service neutron] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:37.407559 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae74d5e-0918-456b-be5d-dfca93822b85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:37.409504 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-5ae74d5e-0918-456b-be5d-dfca93822b85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:37.587487 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae74d5e-0918-456b-be5d-dfca93822b85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:37.588974 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae74d5e-0918-456b-be5d-dfca93822b85 service neutron] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:37.614154 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 836/1658] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:38:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:38:38.601715 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c523564-8c19-43e4-8983-56c7c957d225 tempest-VolumeQuotaClassesTest-320337428 tempest-VolumeQuotaClassesTest-320337428-project-admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:38.606171 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-2c523564-8c19-43e4-8983-56c7c957d225 tempest-VolumeQuotaClassesTest-320337428 tempest-VolumeQuotaClassesTest-320337428-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:38.640521 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c523564-8c19-43e4-8983-56c7c957d225 tempest-VolumeQuotaClassesTest-320337428 tempest-VolumeQuotaClassesTest-320337428-project-admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:38.654937 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5dce77-9026-40c9-b08b-1794a9516033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:38.658032 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5dce77-9026-40c9-b08b-1794a9516033 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:38.658319 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5dce77-9026-40c9-b08b-1794a9516033 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:38.658607 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5dce77-9026-40c9-b08b-1794a9516033 admin admin] PATH_INFO: `/v3/users/dedde243557c450188a0b7cdf9f7b1e7` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:38.660341 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5dce77-9026-40c9-b08b-1794a9516033 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dedde243557c450188a0b7cdf9f7b1e7)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:38.666415 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5dce77-9026-40c9-b08b-1794a9516033 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:38.992969 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-7e5dce77-9026-40c9-b08b-1794a9516033 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dedde243557c450188a0b7cdf9f7b1e7'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:38:38.993873 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-7e5dce77-9026-40c9-b08b-1794a9516033 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dedde243557c450188a0b7cdf9f7b1e7'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:38:39.108864 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-7e5dce77-9026-40c9-b08b-1794a9516033 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dedde243557c450188a0b7cdf9f7b1e7'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:38:39.109540 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-7e5dce77-9026-40c9-b08b-1794a9516033 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dedde243557c450188a0b7cdf9f7b1e7'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:38:39.120709 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-7e5dce77-9026-40c9-b08b-1794a9516033 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dedde243557c450188a0b7cdf9f7b1e7'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:38:39.131939 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae74d5e-0918-456b-be5d-dfca93822b85 service neutron] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:39.134893 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-5ae74d5e-0918-456b-be5d-dfca93822b85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:39.154608 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 823/1659] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:38:38 2022] DELETE /identity/v3/users/dedde243557c450188a0b7cdf9f7b1e7 => generated 0 bytes in 552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:38:39.157238 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4910eb0d-1ead-48ec-9589-b38ff70ce785 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:39.157539 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4910eb0d-1ead-48ec-9589-b38ff70ce785 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:39.157862 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4910eb0d-1ead-48ec-9589-b38ff70ce785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:39.160210 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae74d5e-0918-456b-be5d-dfca93822b85 service neutron] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:39.174228 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:38:39.233908 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-4910eb0d-1ead-48ec-9589-b38ff70ce785 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:38:39.236928 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-4910eb0d-1ead-48ec-9589-b38ff70ce785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:39.300590 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b90e08-af22-43b8-bb7d-cf7b1acbc72a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:39.302053 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b90e08-af22-43b8-bb7d-cf7b1acbc72a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:39.304677 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b90e08-af22-43b8-bb7d-cf7b1acbc72a admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:39.306770 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b90e08-af22-43b8-bb7d-cf7b1acbc72a admin admin] PATH_INFO: `/v3/projects/b17284c066e7495587e5125630248284` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:39.326822 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b90e08-af22-43b8-bb7d-cf7b1acbc72a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b17284c066e7495587e5125630248284)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:39.332008 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b90e08-af22-43b8-bb7d-cf7b1acbc72a admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:39.385672 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 824/1660] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:38:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:38:39.406536 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4910eb0d-1ead-48ec-9589-b38ff70ce785 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:39.407993 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-4910eb0d-1ead-48ec-9589-b38ff70ce785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:39.416310 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:38:39.531976 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-17b90e08-af22-43b8-bb7d-cf7b1acbc72a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b17284c066e7495587e5125630248284', 'request_id': 'req-17b90e08-af22-43b8-bb7d-cf7b1acbc72a'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:38:39.545044 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-17b90e08-af22-43b8-bb7d-cf7b1acbc72a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b17284c066e7495587e5125630248284', 'request_id': 'req-17b90e08-af22-43b8-bb7d-cf7b1acbc72a'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:38:39.567536 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4910eb0d-1ead-48ec-9589-b38ff70ce785 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:39.647868 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-17b90e08-af22-43b8-bb7d-cf7b1acbc72a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b17284c066e7495587e5125630248284', 'request_id': 'req-17b90e08-af22-43b8-bb7d-cf7b1acbc72a'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:38:39.677353 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-17b90e08-af22-43b8-bb7d-cf7b1acbc72a admin admin] The token cache is being invalidate because project b17284c066e7495587e5125630248284 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:38:39.678543 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-17b90e08-af22-43b8-bb7d-cf7b1acbc72a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:38:39.683738 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-17b90e08-af22-43b8-bb7d-cf7b1acbc72a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:38:39.691890 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 837/1661] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:38:39 2022] DELETE /identity/v3/projects/b17284c066e7495587e5125630248284 => generated 0 bytes in 561 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:38:39.697248 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b90e08-af22-43b8-bb7d-cf7b1acbc72a admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:39.704082 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-17b90e08-af22-43b8-bb7d-cf7b1acbc72a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:39.723297 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e841fb-82a7-4e30-b8f6-c75dcb6184f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:39.725932 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e841fb-82a7-4e30-b8f6-c75dcb6184f5 service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:39.725932 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e841fb-82a7-4e30-b8f6-c75dcb6184f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:39.725932 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e841fb-82a7-4e30-b8f6-c75dcb6184f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:39.728158 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-32e841fb-82a7-4e30-b8f6-c75dcb6184f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:39.961699 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e841fb-82a7-4e30-b8f6-c75dcb6184f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:39.963162 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e841fb-82a7-4e30-b8f6-c75dcb6184f5 service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:40.008446 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b90e08-af22-43b8-bb7d-cf7b1acbc72a admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:40.017483 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 825/1662] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:38:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 609 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 07:38:40.017483 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e841fb-82a7-4e30-b8f6-c75dcb6184f5 service neutron] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:40.025609 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a060ab-8494-47da-8d14-f013054e2cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:40.025828 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-32e841fb-82a7-4e30-b8f6-c75dcb6184f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:40.026286 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a060ab-8494-47da-8d14-f013054e2cc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:40.026567 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a060ab-8494-47da-8d14-f013054e2cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:40.026841 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a060ab-8494-47da-8d14-f013054e2cc8 admin admin] PATH_INFO: `/v3/users/5120d3da839c487e917b8bff45044f6e` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:40.043066 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a060ab-8494-47da-8d14-f013054e2cc8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5120d3da839c487e917b8bff45044f6e)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:40.043066 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a060ab-8494-47da-8d14-f013054e2cc8 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:40.076446 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e841fb-82a7-4e30-b8f6-c75dcb6184f5 service neutron] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:40.091428 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56614ed4-f297-4c4a-8f5c-1c05e6a96d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:40.092094 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56614ed4-f297-4c4a-8f5c-1c05e6a96d19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:40.092422 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56614ed4-f297-4c4a-8f5c-1c05e6a96d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:40.092698 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56614ed4-f297-4c4a-8f5c-1c05e6a96d19 admin admin] PATH_INFO: `/v3/users/831b4c254a574c45a760c68be61bd2ef` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:40.095880 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56614ed4-f297-4c4a-8f5c-1c05e6a96d19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=831b4c254a574c45a760c68be61bd2ef)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:40.097455 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56614ed4-f297-4c4a-8f5c-1c05e6a96d19 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:40.351650 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:38:40.355190 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-83a060ab-8494-47da-8d14-f013054e2cc8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5120d3da839c487e917b8bff45044f6e'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:38:40.360977 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-56614ed4-f297-4c4a-8f5c-1c05e6a96d19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '831b4c254a574c45a760c68be61bd2ef'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:38:40.363404 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-56614ed4-f297-4c4a-8f5c-1c05e6a96d19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '831b4c254a574c45a760c68be61bd2ef'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:38:40.418318 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-83a060ab-8494-47da-8d14-f013054e2cc8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5120d3da839c487e917b8bff45044f6e'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:38:40.419041 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-83a060ab-8494-47da-8d14-f013054e2cc8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5120d3da839c487e917b8bff45044f6e'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:38:40.419528 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-83a060ab-8494-47da-8d14-f013054e2cc8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5120d3da839c487e917b8bff45044f6e'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:38:40.433796 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-83a060ab-8494-47da-8d14-f013054e2cc8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5120d3da839c487e917b8bff45044f6e'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:38:40.480937 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 838/1663] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:38:39 2022] DELETE /identity/v3/users/5120d3da839c487e917b8bff45044f6e => generated 0 bytes in 787 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:38:40.486118 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fefc4d-70f5-49de-8b46-0d52a7202ae6 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:40.486586 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fefc4d-70f5-49de-8b46-0d52a7202ae6 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:40.487031 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fefc4d-70f5-49de-8b46-0d52a7202ae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:40.503253 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-56614ed4-f297-4c4a-8f5c-1c05e6a96d19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '831b4c254a574c45a760c68be61bd2ef'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:38:40.504196 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-56614ed4-f297-4c4a-8f5c-1c05e6a96d19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '831b4c254a574c45a760c68be61bd2ef'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:38:40.522740 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-56614ed4-f297-4c4a-8f5c-1c05e6a96d19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '831b4c254a574c45a760c68be61bd2ef'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:38:40.531195 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-05fefc4d-70f5-49de-8b46-0d52a7202ae6 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:38:40.532481 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-05fefc4d-70f5-49de-8b46-0d52a7202ae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:40.564677 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 826/1664] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:38:40 2022] DELETE /identity/v3/users/831b4c254a574c45a760c68be61bd2ef => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:38:40.628978 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 839/1665] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:38:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 07:38:40.705344 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56614ed4-f297-4c4a-8f5c-1c05e6a96d19 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:40.708091 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-56614ed4-f297-4c4a-8f5c-1c05e6a96d19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:40.945156 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:38:40.959923 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56614ed4-f297-4c4a-8f5c-1c05e6a96d19 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:40.968691 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc221ba0-83b7-493c-b3fe-cdda39868089 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:40.971304 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc221ba0-83b7-493c-b3fe-cdda39868089 service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:40.971750 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc221ba0-83b7-493c-b3fe-cdda39868089 service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:40.972918 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc221ba0-83b7-493c-b3fe-cdda39868089 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:41.195002 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc221ba0-83b7-493c-b3fe-cdda39868089 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:41.197338 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc221ba0-83b7-493c-b3fe-cdda39868089 service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:41.226531 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 827/1666] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:38:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:38:41.707012 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fefc4d-70f5-49de-8b46-0d52a7202ae6 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:41.710629 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-05fefc4d-70f5-49de-8b46-0d52a7202ae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:41.724321 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fefc4d-70f5-49de-8b46-0d52a7202ae6 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:41.729452 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db75b677-11c3-4cec-b0ac-338441fd89ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:41.730771 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db75b677-11c3-4cec-b0ac-338441fd89ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:41.730771 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db75b677-11c3-4cec-b0ac-338441fd89ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:41.730956 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db75b677-11c3-4cec-b0ac-338441fd89ed admin admin] PATH_INFO: `/v3/projects/ac0ec290949c4a41ba97f0efc53628d4` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:41.732296 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db75b677-11c3-4cec-b0ac-338441fd89ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac0ec290949c4a41ba97f0efc53628d4)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:41.734193 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db75b677-11c3-4cec-b0ac-338441fd89ed admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:41.803970 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8f5219-76fe-45bf-910f-e3a5ca9b119a None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:41.804586 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8f5219-76fe-45bf-910f-e3a5ca9b119a None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:41.805111 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8f5219-76fe-45bf-910f-e3a5ca9b119a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:41.848360 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-4a8f5219-76fe-45bf-910f-e3a5ca9b119a None None] MFA Rules not processed for user `1d51e6d2afe1436e8a65b895c83bdd38`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:38:41.849905 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-4a8f5219-76fe-45bf-910f-e3a5ca9b119a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:41.883977 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-db75b677-11c3-4cec-b0ac-338441fd89ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac0ec290949c4a41ba97f0efc53628d4', 'request_id': 'req-db75b677-11c3-4cec-b0ac-338441fd89ed'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:38:41.905954 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-db75b677-11c3-4cec-b0ac-338441fd89ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac0ec290949c4a41ba97f0efc53628d4', 'request_id': 'req-db75b677-11c3-4cec-b0ac-338441fd89ed'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:38:41.967366 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 828/1667] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:38:41 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 07:38:41.977215 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef788d1-1be7-4ea5-b76b-f44ade99d778 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:41.977631 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef788d1-1be7-4ea5-b76b-f44ade99d778 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:41.978062 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef788d1-1be7-4ea5-b76b-f44ade99d778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:42.001031 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-3ef788d1-1be7-4ea5-b76b-f44ade99d778 None None] MFA Rules not processed for user `1d51e6d2afe1436e8a65b895c83bdd38`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:38:42.002436 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-3ef788d1-1be7-4ea5-b76b-f44ade99d778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:42.019066 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-db75b677-11c3-4cec-b0ac-338441fd89ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac0ec290949c4a41ba97f0efc53628d4', 'request_id': 'req-db75b677-11c3-4cec-b0ac-338441fd89ed'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:38:42.075289 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-db75b677-11c3-4cec-b0ac-338441fd89ed admin admin] The token cache is being invalidate because project ac0ec290949c4a41ba97f0efc53628d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:38:42.076599 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-db75b677-11c3-4cec-b0ac-338441fd89ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:38:42.079501 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-db75b677-11c3-4cec-b0ac-338441fd89ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:38:42.088223 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 840/1668] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:38:41 2022] DELETE /identity/v3/projects/ac0ec290949c4a41ba97f0efc53628d4 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:38:42.094387 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 829/1669] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:38:41 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:38:42.110712 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db75b677-11c3-4cec-b0ac-338441fd89ed admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:42.110896 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef788d1-1be7-4ea5-b76b-f44ade99d778 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:42.112606 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-db75b677-11c3-4cec-b0ac-338441fd89ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:42.113195 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-3ef788d1-1be7-4ea5-b76b-f44ade99d778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:42.247423 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:38:42.268353 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db75b677-11c3-4cec-b0ac-338441fd89ed admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:42.275994 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e936adef-eb13-410b-a52d-4e79436cf3fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:42.277197 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e936adef-eb13-410b-a52d-4e79436cf3fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:42.277496 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e936adef-eb13-410b-a52d-4e79436cf3fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:42.284734 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e936adef-eb13-410b-a52d-4e79436cf3fc admin admin] PATH_INFO: `/v3/users/3549490d84fe4650a39e3859fc919a72` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:42.284734 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e936adef-eb13-410b-a52d-4e79436cf3fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=3549490d84fe4650a39e3859fc919a72)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:42.290194 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e936adef-eb13-410b-a52d-4e79436cf3fc admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:42.336611 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef788d1-1be7-4ea5-b76b-f44ade99d778 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:42.343496 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca18706a-5ce7-43fd-a517-fb5b65e24aa8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a080c7ab9b684abca6828067f2572c3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:42.347521 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca18706a-5ce7-43fd-a517-fb5b65e24aa8 service nova] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:42.347521 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca18706a-5ce7-43fd-a517-fb5b65e24aa8 service nova] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:42.347521 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca18706a-5ce7-43fd-a517-fb5b65e24aa8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:42.347521 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-ca18706a-5ce7-43fd-a517-fb5b65e24aa8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:42.361023 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca18706a-5ce7-43fd-a517-fb5b65e24aa8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:42.363278 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca18706a-5ce7-43fd-a517-fb5b65e24aa8 service nova] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:42.388706 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 830/1670] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:38:42 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:38:42.567082 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-e936adef-eb13-410b-a52d-4e79436cf3fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3549490d84fe4650a39e3859fc919a72'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:38:42.663102 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-e936adef-eb13-410b-a52d-4e79436cf3fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3549490d84fe4650a39e3859fc919a72'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:38:42.668575 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-e936adef-eb13-410b-a52d-4e79436cf3fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3549490d84fe4650a39e3859fc919a72'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:38:42.670199 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-e936adef-eb13-410b-a52d-4e79436cf3fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3549490d84fe4650a39e3859fc919a72'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:38:42.687026 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-e936adef-eb13-410b-a52d-4e79436cf3fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3549490d84fe4650a39e3859fc919a72'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:38:42.723041 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 841/1671] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:38:42 2022] DELETE /identity/v3/users/3549490d84fe4650a39e3859fc919a72 => generated 0 bytes in 613 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:38:42.848685 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:38:43.316554 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b1789-78c9-4987-96a7-242e9d18402f None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:43.317187 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b1789-78c9-4987-96a7-242e9d18402f None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:43.317470 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b1789-78c9-4987-96a7-242e9d18402f None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:43.319116 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 831/1672] 10.222.0.42 () {54 vars in 912 bytes} [Thu Sep 15 07:38:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:38:43.325341 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40641b8f-f8ce-4c70-a7f8-59d1ecb2e00c None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:43.325811 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40641b8f-f8ce-4c70-a7f8-59d1ecb2e00c None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:43.326184 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40641b8f-f8ce-4c70-a7f8-59d1ecb2e00c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:43.405559 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-40641b8f-f8ce-4c70-a7f8-59d1ecb2e00c None None] MFA Rules not processed for user `5315996432f44bffa3460b2744c10f0a`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:38:43.407234 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-40641b8f-f8ce-4c70-a7f8-59d1ecb2e00c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:43.508490 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 842/1673] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Sep 15 07:38:43 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:38:43.514644 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6b1789-78c9-4987-96a7-242e9d18402f None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:43.518419 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-3c6b1789-78c9-4987-96a7-242e9d18402f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:43.526658 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40641b8f-f8ce-4c70-a7f8-59d1ecb2e00c None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:43.529827 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-40641b8f-f8ce-4c70-a7f8-59d1ecb2e00c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:43.550643 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40641b8f-f8ce-4c70-a7f8-59d1ecb2e00c None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:43.565953 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff29556a-badc-484f-8d2f-b244609b2d3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:43.570464 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff29556a-badc-484f-8d2f-b244609b2d3c service neutron] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:43.570755 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff29556a-badc-484f-8d2f-b244609b2d3c service neutron] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:43.571089 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff29556a-badc-484f-8d2f-b244609b2d3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:43.592978 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff29556a-badc-484f-8d2f-b244609b2d3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:43.595195 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff29556a-badc-484f-8d2f-b244609b2d3c service neutron] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:43.625935 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 843/1674] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:38:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:38:43.626925 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff29556a-badc-484f-8d2f-b244609b2d3c service neutron] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:43.628667 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-ff29556a-badc-484f-8d2f-b244609b2d3c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:43.722516 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6b1789-78c9-4987-96a7-242e9d18402f None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:43.738902 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b07bce-4cf1-4069-adad-99210c398e8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:43.742862 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b07bce-4cf1-4069-adad-99210c398e8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:43.743477 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b07bce-4cf1-4069-adad-99210c398e8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:43.743935 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b07bce-4cf1-4069-adad-99210c398e8f admin admin] PATH_INFO: `/v3/projects/a22520811f1845d797ad8721a0e8f4e8` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:43.753597 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b07bce-4cf1-4069-adad-99210c398e8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a22520811f1845d797ad8721a0e8f4e8)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:43.755788 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b07bce-4cf1-4069-adad-99210c398e8f admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:43.783809 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff29556a-badc-484f-8d2f-b244609b2d3c service neutron] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:43.792203 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f15978-5007-456b-8cb0-7652e4669003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:43.793152 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f15978-5007-456b-8cb0-7652e4669003 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:43.793568 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f15978-5007-456b-8cb0-7652e4669003 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:43.794063 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f15978-5007-456b-8cb0-7652e4669003 admin admin] PATH_INFO: `/v3/projects/3dfbf0bb45d9413296e05adb32a63fb0` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:43.795501 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f15978-5007-456b-8cb0-7652e4669003 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3dfbf0bb45d9413296e05adb32a63fb0)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:43.798103 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f15978-5007-456b-8cb0-7652e4669003 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:43.965055 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-91b07bce-4cf1-4069-adad-99210c398e8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a22520811f1845d797ad8721a0e8f4e8', 'request_id': 'req-91b07bce-4cf1-4069-adad-99210c398e8f'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:38:43.980205 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-91b07bce-4cf1-4069-adad-99210c398e8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a22520811f1845d797ad8721a0e8f4e8', 'request_id': 'req-91b07bce-4cf1-4069-adad-99210c398e8f'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:38:44.015084 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-85f15978-5007-456b-8cb0-7652e4669003 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3dfbf0bb45d9413296e05adb32a63fb0', 'request_id': 'req-85f15978-5007-456b-8cb0-7652e4669003'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:38:44.027140 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-85f15978-5007-456b-8cb0-7652e4669003 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3dfbf0bb45d9413296e05adb32a63fb0', 'request_id': 'req-85f15978-5007-456b-8cb0-7652e4669003'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:38:44.063534 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-91b07bce-4cf1-4069-adad-99210c398e8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a22520811f1845d797ad8721a0e8f4e8', 'request_id': 'req-91b07bce-4cf1-4069-adad-99210c398e8f'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:38:44.086897 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-91b07bce-4cf1-4069-adad-99210c398e8f admin admin] The token cache is being invalidate because project a22520811f1845d797ad8721a0e8f4e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:38:44.087395 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-91b07bce-4cf1-4069-adad-99210c398e8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:38:44.088444 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-91b07bce-4cf1-4069-adad-99210c398e8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:38:44.092474 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 832/1675] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:38:43 2022] DELETE /identity/v3/projects/a22520811f1845d797ad8721a0e8f4e8 => generated 0 bytes in 578 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:38:44.098276 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4595bed6-f768-48e0-a0f3-aa4ba07170d2 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:44.098880 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4595bed6-f768-48e0-a0f3-aa4ba07170d2 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:44.099296 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4595bed6-f768-48e0-a0f3-aa4ba07170d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:44.132155 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-4595bed6-f768-48e0-a0f3-aa4ba07170d2 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:38:44.133771 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-4595bed6-f768-48e0-a0f3-aa4ba07170d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:44.143758 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-85f15978-5007-456b-8cb0-7652e4669003 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3dfbf0bb45d9413296e05adb32a63fb0', 'request_id': 'req-85f15978-5007-456b-8cb0-7652e4669003'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:38:44.156747 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:38:44.163943 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-85f15978-5007-456b-8cb0-7652e4669003 admin admin] The token cache is being invalidate because project 3dfbf0bb45d9413296e05adb32a63fb0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:38:44.164239 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-85f15978-5007-456b-8cb0-7652e4669003 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:38:44.165718 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-85f15978-5007-456b-8cb0-7652e4669003 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:38:44.168933 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 844/1676] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:38:43 2022] DELETE /identity/v3/projects/3dfbf0bb45d9413296e05adb32a63fb0 => generated 0 bytes in 542 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:38:44.175065 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c88f86-02af-4af5-9b83-b2d942a688ce None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:44.175702 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c88f86-02af-4af5-9b83-b2d942a688ce None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:44.176237 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c88f86-02af-4af5-9b83-b2d942a688ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:44.228934 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-29c88f86-02af-4af5-9b83-b2d942a688ce None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:38:44.233533 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-29c88f86-02af-4af5-9b83-b2d942a688ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:44.236727 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:38:44.296142 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 833/1677] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:38:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 07:38:44.305178 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a71f5-98ec-4549-95e3-6aec5e5bc52d None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:44.306966 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a71f5-98ec-4549-95e3-6aec5e5bc52d None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:44.312215 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a71f5-98ec-4549-95e3-6aec5e5bc52d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:44.351710 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 845/1678] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:38:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:38:44.354689 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ce9fc8-b76f-420d-9c37-c8ec2c14a7db None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:44.355182 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ce9fc8-b76f-420d-9c37-c8ec2c14a7db None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:44.355594 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ce9fc8-b76f-420d-9c37-c8ec2c14a7db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:44.364906 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-437a71f5-98ec-4549-95e3-6aec5e5bc52d None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:38:44.370511 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-437a71f5-98ec-4549-95e3-6aec5e5bc52d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:44.379921 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-71ce9fc8-b76f-420d-9c37-c8ec2c14a7db None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:38:44.382514 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-71ce9fc8-b76f-420d-9c37-c8ec2c14a7db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:44.388806 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 834/1679] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:38:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:38:44.390168 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437a71f5-98ec-4549-95e3-6aec5e5bc52d None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:44.401732 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 846/1680] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:38:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:38:44.402930 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ce9fc8-b76f-420d-9c37-c8ec2c14a7db None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:44.415698 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ce9fc8-b76f-420d-9c37-c8ec2c14a7db None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:44.416952 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437a71f5-98ec-4549-95e3-6aec5e5bc52d None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:44.421825 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e5a9d0-20ab-49df-be61-feab130184b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:44.422680 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e5a9d0-20ab-49df-be61-feab130184b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:44.422843 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64276932-138c-4ead-b874-9e779a9e9925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:44.423312 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e5a9d0-20ab-49df-be61-feab130184b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:44.423862 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64276932-138c-4ead-b874-9e779a9e9925 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:44.424049 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e5a9d0-20ab-49df-be61-feab130184b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:44.424310 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64276932-138c-4ead-b874-9e779a9e9925 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:44.424677 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64276932-138c-4ead-b874-9e779a9e9925 admin admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:44.425271 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e5a9d0-20ab-49df-be61-feab130184b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:38:44.425556 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64276932-138c-4ead-b874-9e779a9e9925 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:38:44.425891 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e5a9d0-20ab-49df-be61-feab130184b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:44.425993 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64276932-138c-4ead-b874-9e779a9e9925 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:44.428460 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e5a9d0-20ab-49df-be61-feab130184b1 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:44.431114 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64276932-138c-4ead-b874-9e779a9e9925 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:44.451948 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 847/1681] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 15 07:38:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:38:44.451948 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e5a9d0-20ab-49df-be61-feab130184b1 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:44.456592 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 835/1682] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 15 07:38:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:38:44.457645 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64276932-138c-4ead-b874-9e779a9e9925 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:44.472206 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e5a9d0-20ab-49df-be61-feab130184b1 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:44.475575 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64276932-138c-4ead-b874-9e779a9e9925 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:44.488579 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2547435a-be8a-4d78-9101-0338bdbe0a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:44.488945 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dc3b97-c2ca-4519-b61d-e7817071dbfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:44.489920 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2547435a-be8a-4d78-9101-0338bdbe0a73 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:44.490092 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dc3b97-c2ca-4519-b61d-e7817071dbfb admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:44.490323 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2547435a-be8a-4d78-9101-0338bdbe0a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:44.490811 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dc3b97-c2ca-4519-b61d-e7817071dbfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:44.492657 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dc3b97-c2ca-4519-b61d-e7817071dbfb admin admin] PATH_INFO: `/v3/projects` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:44.493563 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dc3b97-c2ca-4519-b61d-e7817071dbfb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:44.495826 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2547435a-be8a-4d78-9101-0338bdbe0a73 admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:44.496044 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dc3b97-c2ca-4519-b61d-e7817071dbfb admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:44.498312 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2547435a-be8a-4d78-9101-0338bdbe0a73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:44.504260 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2547435a-be8a-4d78-9101-0338bdbe0a73 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:44.654118 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 848/1683] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:38:44 2022] POST /identity/v3/projects => generated 367 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:38:44.664263 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dc3b97-c2ca-4519-b61d-e7817071dbfb admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:44.672948 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 836/1684] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:38:44 2022] POST /identity/v3/projects => generated 355 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:38:44.672948 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2547435a-be8a-4d78-9101-0338bdbe0a73 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:44.870153 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dc3b97-c2ca-4519-b61d-e7817071dbfb admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:44.879967 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2547435a-be8a-4d78-9101-0338bdbe0a73 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:44.881054 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26fe991-5e3b-42b6-8933-153fd18aed11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:44.884290 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26fe991-5e3b-42b6-8933-153fd18aed11 admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:44.884740 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26fe991-5e3b-42b6-8933-153fd18aed11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:44.885158 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26fe991-5e3b-42b6-8933-153fd18aed11 admin admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:44.885995 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26fe991-5e3b-42b6-8933-153fd18aed11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:44.888339 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26fe991-5e3b-42b6-8933-153fd18aed11 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:44.888897 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510bbde8-5bf8-44d8-91c8-f09932861f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:44.889755 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510bbde8-5bf8-44d8-91c8-f09932861f5a admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:44.889990 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510bbde8-5bf8-44d8-91c8-f09932861f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:44.890268 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510bbde8-5bf8-44d8-91c8-f09932861f5a admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:44.890921 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510bbde8-5bf8-44d8-91c8-f09932861f5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:44.893058 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510bbde8-5bf8-44d8-91c8-f09932861f5a admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:44.989362 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 849/1685] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:38:44 2022] POST /identity/v3/users => generated 284 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:38:44.989362 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26fe991-5e3b-42b6-8933-153fd18aed11 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:44.992727 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-c26fe991-5e3b-42b6-8933-153fd18aed11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:45.017008 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26fe991-5e3b-42b6-8933-153fd18aed11 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:45.019209 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 837/1686] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:38:44 2022] POST /identity/v3/users => generated 278 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:38:45.019428 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510bbde8-5bf8-44d8-91c8-f09932861f5a admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:45.023183 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b09ed4-33a0-4fcc-8261-0598e05c1380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:45.024094 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b09ed4-33a0-4fcc-8261-0598e05c1380 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:45.024533 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b09ed4-33a0-4fcc-8261-0598e05c1380 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:45.024979 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b09ed4-33a0-4fcc-8261-0598e05c1380 admin admin] PATH_INFO: `/v3/projects/c59b01df6dd049918fc0fa0cbdedd89f` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:45.026308 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b09ed4-33a0-4fcc-8261-0598e05c1380 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c59b01df6dd049918fc0fa0cbdedd89f)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:45.028585 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b09ed4-33a0-4fcc-8261-0598e05c1380 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:45.028871 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-510bbde8-5bf8-44d8-91c8-f09932861f5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:45.108706 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:38:45.132012 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:INSERT=5,keystone:DELETE=3,keystone:SELECT=92 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:38:45.198121 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510bbde8-5bf8-44d8-91c8-f09932861f5a admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:45.229367 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26028ddf-617e-4aff-a5ba-27ac6f763ec1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd87846b9a134248b28e0521b0a880c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:45.231892 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26028ddf-617e-4aff-a5ba-27ac6f763ec1 service placement] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:45.232171 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26028ddf-617e-4aff-a5ba-27ac6f763ec1 service placement] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:45.232590 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26028ddf-617e-4aff-a5ba-27ac6f763ec1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:45.239892 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-26028ddf-617e-4aff-a5ba-27ac6f763ec1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:45.272653 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26028ddf-617e-4aff-a5ba-27ac6f763ec1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:45.278955 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-e6b09ed4-33a0-4fcc-8261-0598e05c1380 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c59b01df6dd049918fc0fa0cbdedd89f', 'request_id': 'req-e6b09ed4-33a0-4fcc-8261-0598e05c1380'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:38:45.281831 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26028ddf-617e-4aff-a5ba-27ac6f763ec1 service placement] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:45.306967 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-e6b09ed4-33a0-4fcc-8261-0598e05c1380 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c59b01df6dd049918fc0fa0cbdedd89f', 'request_id': 'req-e6b09ed4-33a0-4fcc-8261-0598e05c1380'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:38:45.322695 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 838/1687] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:38:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:38:45.323665 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26028ddf-617e-4aff-a5ba-27ac6f763ec1 service placement] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:45.353332 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26028ddf-617e-4aff-a5ba-27ac6f763ec1 service placement] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:45.427428 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-e6b09ed4-33a0-4fcc-8261-0598e05c1380 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c59b01df6dd049918fc0fa0cbdedd89f', 'request_id': 'req-e6b09ed4-33a0-4fcc-8261-0598e05c1380'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:38:45.461614 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-e6b09ed4-33a0-4fcc-8261-0598e05c1380 admin admin] The token cache is being invalidate because project c59b01df6dd049918fc0fa0cbdedd89f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:38:45.462026 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-e6b09ed4-33a0-4fcc-8261-0598e05c1380 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:38:45.468880 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-e6b09ed4-33a0-4fcc-8261-0598e05c1380 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:38:45.474058 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 850/1688] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:38:44 2022] DELETE /identity/v3/projects/c59b01df6dd049918fc0fa0cbdedd89f => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:38:45.477829 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b09ed4-33a0-4fcc-8261-0598e05c1380 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:45.478798 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-e6b09ed4-33a0-4fcc-8261-0598e05c1380 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:45.511231 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06026f3f-82e5-4168-91c8-841d6d4c0b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:45.511887 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06026f3f-82e5-4168-91c8-841d6d4c0b7e admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:45.512659 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06026f3f-82e5-4168-91c8-841d6d4c0b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:45.512659 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06026f3f-82e5-4168-91c8-841d6d4c0b7e admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:45.513110 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06026f3f-82e5-4168-91c8-841d6d4c0b7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:45.515100 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06026f3f-82e5-4168-91c8-841d6d4c0b7e admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:45.535342 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 839/1689] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:38:45 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:38:45.537307 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5863e64b-ed8d-4789-b19b-af294cc05153 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:45.537836 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5863e64b-ed8d-4789-b19b-af294cc05153 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:45.538258 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5863e64b-ed8d-4789-b19b-af294cc05153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:45.569567 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-5863e64b-ed8d-4789-b19b-af294cc05153 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:38:45.570334 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-5863e64b-ed8d-4789-b19b-af294cc05153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:45.582946 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 840/1690] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:38:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:38:45.583949 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5863e64b-ed8d-4789-b19b-af294cc05153 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:45.588941 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-5863e64b-ed8d-4789-b19b-af294cc05153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:45.611423 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5863e64b-ed8d-4789-b19b-af294cc05153 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:45.626543 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b09ed4-33a0-4fcc-8261-0598e05c1380 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:45.627493 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4cbc09-3a49-44ad-b411-a12e5c7d578b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:45.628361 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4cbc09-3a49-44ad-b411-a12e5c7d578b admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:45.628646 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4cbc09-3a49-44ad-b411-a12e5c7d578b admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:45.628917 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4cbc09-3a49-44ad-b411-a12e5c7d578b admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:45.629737 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4cbc09-3a49-44ad-b411-a12e5c7d578b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:45.631413 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4cbc09-3a49-44ad-b411-a12e5c7d578b admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:45.644456 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8b803f-1b2e-4af7-b158-aa8150e12a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:45.645193 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8b803f-1b2e-4af7-b158-aa8150e12a84 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:45.645477 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8b803f-1b2e-4af7-b158-aa8150e12a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:45.646039 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8b803f-1b2e-4af7-b158-aa8150e12a84 admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:45.646693 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8b803f-1b2e-4af7-b158-aa8150e12a84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:45.649284 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8b803f-1b2e-4af7-b158-aa8150e12a84 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:45.653555 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 841/1691] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:38:45 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:38:45.656055 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1505b3b8-3b57-44ce-a8f4-983896009fd2 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:45.656348 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1505b3b8-3b57-44ce-a8f4-983896009fd2 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:45.656624 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1505b3b8-3b57-44ce-a8f4-983896009fd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:45.677887 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 851/1692] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:38:45 2022] GET /identity/v3/roles => generated 1420 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:38:45.680912 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8b803f-1b2e-4af7-b158-aa8150e12a84 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:45.706704 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-1505b3b8-3b57-44ce-a8f4-983896009fd2 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:38:45.707941 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-1505b3b8-3b57-44ce-a8f4-983896009fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:45.712100 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8b803f-1b2e-4af7-b158-aa8150e12a84 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:45.719936 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f766af1-ee38-474c-bf50-211b2a352a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:45.721226 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f766af1-ee38-474c-bf50-211b2a352a0c admin admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:45.721860 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f766af1-ee38-474c-bf50-211b2a352a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:45.730009 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 842/1693] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:38:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:38:45.730896 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1505b3b8-3b57-44ce-a8f4-983896009fd2 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:45.737954 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f766af1-ee38-474c-bf50-211b2a352a0c admin admin] PATH_INFO: `/v3/projects/dee2984dda404692a5058904534b449c/users/c2cb219f5ffa426b9f098859681e1fb3/roles/053e12af84e943e699460a9d76c2072b` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:45.755041 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1505b3b8-3b57-44ce-a8f4-983896009fd2 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:45.765012 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3a7cd4-5884-41f6-ac37-00dca9c82170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:45.766092 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3a7cd4-5884-41f6-ac37-00dca9c82170 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:45.766576 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3a7cd4-5884-41f6-ac37-00dca9c82170 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:45.767070 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3a7cd4-5884-41f6-ac37-00dca9c82170 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:45.767928 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3a7cd4-5884-41f6-ac37-00dca9c82170 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:45.771418 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3a7cd4-5884-41f6-ac37-00dca9c82170 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:45.786270 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f766af1-ee38-474c-bf50-211b2a352a0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=dee2984dda404692a5058904534b449c, user_id=c2cb219f5ffa426b9f098859681e1fb3, role_id=053e12af84e943e699460a9d76c2072b)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:45.797998 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f766af1-ee38-474c-bf50-211b2a352a0c admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:45.801834 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 843/1694] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:38:45 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:38:45.801834 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3a7cd4-5884-41f6-ac37-00dca9c82170 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:45.811818 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3a7cd4-5884-41f6-ac37-00dca9c82170 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:45.820327 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb087577-b860-41c5-a94f-de04f47f68b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:45.821254 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb087577-b860-41c5-a94f-de04f47f68b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:45.822047 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb087577-b860-41c5-a94f-de04f47f68b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:45.822556 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb087577-b860-41c5-a94f-de04f47f68b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:45.823578 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb087577-b860-41c5-a94f-de04f47f68b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:38:45.824117 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb087577-b860-41c5-a94f-de04f47f68b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:45.826088 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb087577-b860-41c5-a94f-de04f47f68b7 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:45.851551 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 844/1695] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 15 07:38:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:38:45.852628 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb087577-b860-41c5-a94f-de04f47f68b7 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:45.864857 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb087577-b860-41c5-a94f-de04f47f68b7 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:45.895364 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 852/1696] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:38:45 2022] PUT /identity/v3/projects/dee2984dda404692a5058904534b449c/users/c2cb219f5ffa426b9f098859681e1fb3/roles/053e12af84e943e699460a9d76c2072b => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:38:45.895364 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f766af1-ee38-474c-bf50-211b2a352a0c admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:45.903303 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b521068b-6828-401a-b82d-de246696877b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:45.904581 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b521068b-6828-401a-b82d-de246696877b admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:45.905708 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b521068b-6828-401a-b82d-de246696877b admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:45.906031 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b521068b-6828-401a-b82d-de246696877b admin admin] PATH_INFO: `/v3/projects/7717ab5e49114e9b88999bb5dcd8df45/users/7c804d651b8149f99718c576e7c187c7/roles/053e12af84e943e699460a9d76c2072b` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:45.928273 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:38:45.933159 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b521068b-6828-401a-b82d-de246696877b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7717ab5e49114e9b88999bb5dcd8df45, user_id=7c804d651b8149f99718c576e7c187c7, role_id=053e12af84e943e699460a9d76c2072b)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:45.936520 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b521068b-6828-401a-b82d-de246696877b admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:46.049601 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 845/1697] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:38:45 2022] PUT /identity/v3/projects/7717ab5e49114e9b88999bb5dcd8df45/users/7c804d651b8149f99718c576e7c187c7/roles/053e12af84e943e699460a9d76c2072b => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:38:46.049601 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834c1ca9-dd3e-4b0c-89b3-97feb4772c84 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:46.049601 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834c1ca9-dd3e-4b0c-89b3-97feb4772c84 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:46.049601 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834c1ca9-dd3e-4b0c-89b3-97feb4772c84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:46.053221 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f766af1-ee38-474c-bf50-211b2a352a0c admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:46.095145 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-834c1ca9-dd3e-4b0c-89b3-97feb4772c84 None None] MFA Rules not processed for user `c2cb219f5ffa426b9f098859681e1fb3`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:38:46.097618 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-834c1ca9-dd3e-4b0c-89b3-97feb4772c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:46.184397 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:38:46.207928 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d3f6f2-aba3-4b6e-8885-302d9fd24963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:46.208462 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d3f6f2-aba3-4b6e-8885-302d9fd24963 admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:46.208940 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d3f6f2-aba3-4b6e-8885-302d9fd24963 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:46.209286 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d3f6f2-aba3-4b6e-8885-302d9fd24963 admin admin] PATH_INFO: `/v3/projects` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:46.210854 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d3f6f2-aba3-4b6e-8885-302d9fd24963 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:46.214075 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d3f6f2-aba3-4b6e-8885-302d9fd24963 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:46.235093 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 846/1698] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:38:46 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:38:46.237095 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c113fa85-0bf8-4b13-80a6-d47d9eadab2d None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:46.237532 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c113fa85-0bf8-4b13-80a6-d47d9eadab2d None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:46.238000 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c113fa85-0bf8-4b13-80a6-d47d9eadab2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:46.277458 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-c113fa85-0bf8-4b13-80a6-d47d9eadab2d None None] MFA Rules not processed for user `7c804d651b8149f99718c576e7c187c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:38:46.278748 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-c113fa85-0bf8-4b13-80a6-d47d9eadab2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:46.328634 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 853/1699] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:38:45 2022] POST /identity/v3/projects => generated 393 bytes in 442 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:38:46.330543 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466e80b3-7381-4af4-a92f-325fca96d02f None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:46.330937 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466e80b3-7381-4af4-a92f-325fca96d02f None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:46.331339 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466e80b3-7381-4af4-a92f-325fca96d02f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:46.456323 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-466e80b3-7381-4af4-a92f-325fca96d02f None None] MFA Rules not processed for user `5e0c8d7aa4db455a94631becf212866f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:38:46.466485 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-466e80b3-7381-4af4-a92f-325fca96d02f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:46.553401 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 847/1700] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:38:46 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:38:46.553401 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c113fa85-0bf8-4b13-80a6-d47d9eadab2d None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:46.560596 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-c113fa85-0bf8-4b13-80a6-d47d9eadab2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:46.702779 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c113fa85-0bf8-4b13-80a6-d47d9eadab2d None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:46.708757 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 854/1701] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:38:46 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 377 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:38:46.708757 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466e80b3-7381-4af4-a92f-325fca96d02f None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:46.817390 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:38:46.839799 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466e80b3-7381-4af4-a92f-325fca96d02f None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:46.847151 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9714f6d0-31bd-4920-a076-a9bf9cebf6d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:46.849393 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9714f6d0-31bd-4920-a076-a9bf9cebf6d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:46.857961 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9714f6d0-31bd-4920-a076-a9bf9cebf6d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:46.858623 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9714f6d0-31bd-4920-a076-a9bf9cebf6d2 admin admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:46.859670 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9714f6d0-31bd-4920-a076-a9bf9cebf6d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:46.861844 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9714f6d0-31bd-4920-a076-a9bf9cebf6d2 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:46.891268 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edff6f32-0694-47a8-a9fa-a1341ed8311a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecf6642b7b84d3c9756bb41ab3349d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:46.891915 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff6f32-0694-47a8-a9fa-a1341ed8311a service cinder] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:46.892195 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff6f32-0694-47a8-a9fa-a1341ed8311a service cinder] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:46.892479 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff6f32-0694-47a8-a9fa-a1341ed8311a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:46.955773 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:38:46.967675 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 855/1702] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:38:46 2022] POST /identity/v3/users => generated 297 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:38:46.971320 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9714f6d0-31bd-4920-a076-a9bf9cebf6d2 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:46.996686 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edff6f32-0694-47a8-a9fa-a1341ed8311a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:46.998228 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9714f6d0-31bd-4920-a076-a9bf9cebf6d2 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:47.005680 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edff6f32-0694-47a8-a9fa-a1341ed8311a service cinder] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:47.008752 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396cc48d-bc5f-45ed-803b-d1f495b6b819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:47.010333 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396cc48d-bc5f-45ed-803b-d1f495b6b819 admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:47.010622 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396cc48d-bc5f-45ed-803b-d1f495b6b819 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:47.011071 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396cc48d-bc5f-45ed-803b-d1f495b6b819 admin admin] PATH_INFO: `/v3/projects` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:47.011903 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396cc48d-bc5f-45ed-803b-d1f495b6b819 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:47.013920 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396cc48d-bc5f-45ed-803b-d1f495b6b819 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:47.035824 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 848/1703] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:38:46 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:38:47.037828 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edff6f32-0694-47a8-a9fa-a1341ed8311a service cinder] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:47.042115 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-edff6f32-0694-47a8-a9fa-a1341ed8311a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:47.158442 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 856/1704] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:38:46 2022] POST /identity/v3/projects => generated 357 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:38:47.159553 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396cc48d-bc5f-45ed-803b-d1f495b6b819 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:47.162683 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-396cc48d-bc5f-45ed-803b-d1f495b6b819 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:47.307516 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edff6f32-0694-47a8-a9fa-a1341ed8311a service cinder] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:47.359906 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396cc48d-bc5f-45ed-803b-d1f495b6b819 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:47.375101 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f29a3d-ecbf-4f3c-ae8b-bc73ca5e21af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:47.375886 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f29a3d-ecbf-4f3c-ae8b-bc73ca5e21af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:47.376258 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f29a3d-ecbf-4f3c-ae8b-bc73ca5e21af admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:47.376629 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f29a3d-ecbf-4f3c-ae8b-bc73ca5e21af admin admin] PATH_INFO: `/v3/users/370896120c7d4fc7b0bf9548336f9382` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:47.378229 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f29a3d-ecbf-4f3c-ae8b-bc73ca5e21af admin admin] RBAC: Authorizing `identity:delete_user(user_id=370896120c7d4fc7b0bf9548336f9382)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:47.384106 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f29a3d-ecbf-4f3c-ae8b-bc73ca5e21af admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:47.478426 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7659bc95-ac9b-40af-b751-2b472741b09a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a080c7ab9b684abca6828067f2572c3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:47.480142 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7659bc95-ac9b-40af-b751-2b472741b09a service nova] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:47.480429 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7659bc95-ac9b-40af-b751-2b472741b09a service nova] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:47.480737 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7659bc95-ac9b-40af-b751-2b472741b09a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:47.595380 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7659bc95-ac9b-40af-b751-2b472741b09a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:47.606964 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7659bc95-ac9b-40af-b751-2b472741b09a service nova] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:47.638153 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 849/1705] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:38:47 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 591 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:38:47.646517 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7659bc95-ac9b-40af-b751-2b472741b09a service nova] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:47.691337 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7659bc95-ac9b-40af-b751-2b472741b09a service nova] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:47.695799 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-a9f29a3d-ecbf-4f3c-ae8b-bc73ca5e21af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '370896120c7d4fc7b0bf9548336f9382'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:38:47.704344 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7e306f-fa86-44a0-9242-96082a3cd451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:47.704981 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7e306f-fa86-44a0-9242-96082a3cd451 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:47.705264 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7e306f-fa86-44a0-9242-96082a3cd451 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:47.705537 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7e306f-fa86-44a0-9242-96082a3cd451 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:47.707118 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7e306f-fa86-44a0-9242-96082a3cd451 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:47.710438 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7e306f-fa86-44a0-9242-96082a3cd451 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:47.716303 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:38:47.740771 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 850/1706] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:38:47 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:38:47.742631 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7e306f-fa86-44a0-9242-96082a3cd451 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:47.758284 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7e306f-fa86-44a0-9242-96082a3cd451 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:47.768001 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad33bbf-e25c-464e-aee8-7166877ca8eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:47.768657 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad33bbf-e25c-464e-aee8-7166877ca8eb admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:47.769024 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad33bbf-e25c-464e-aee8-7166877ca8eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:47.769477 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad33bbf-e25c-464e-aee8-7166877ca8eb admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:47.770244 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad33bbf-e25c-464e-aee8-7166877ca8eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:47.774751 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad33bbf-e25c-464e-aee8-7166877ca8eb admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:47.802672 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-a9f29a3d-ecbf-4f3c-ae8b-bc73ca5e21af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '370896120c7d4fc7b0bf9548336f9382'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:38:47.806934 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-a9f29a3d-ecbf-4f3c-ae8b-bc73ca5e21af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '370896120c7d4fc7b0bf9548336f9382'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:38:47.810211 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-a9f29a3d-ecbf-4f3c-ae8b-bc73ca5e21af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '370896120c7d4fc7b0bf9548336f9382'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:38:47.839006 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-a9f29a3d-ecbf-4f3c-ae8b-bc73ca5e21af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '370896120c7d4fc7b0bf9548336f9382'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:38:47.888845 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 851/1707] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:38:47 2022] POST /identity/v3/users => generated 285 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:38:47.890543 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaa437a-d9f5-4df3-895b-dc8d67aad0a1 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:47.890793 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaa437a-d9f5-4df3-895b-dc8d67aad0a1 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:47.891070 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaa437a-d9f5-4df3-895b-dc8d67aad0a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:47.942174 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 857/1708] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:38:47 2022] DELETE /identity/v3/users/370896120c7d4fc7b0bf9548336f9382 => generated 0 bytes in 767 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:38:47.942174 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f29a3d-ecbf-4f3c-ae8b-bc73ca5e21af admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:47.943479 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-a9f29a3d-ecbf-4f3c-ae8b-bc73ca5e21af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:47.966344 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-3aaa437a-d9f5-4df3-895b-dc8d67aad0a1 None None] MFA Rules not processed for user `56c2f854cf074b1ba2571766b7f2fc51`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:38:47.968069 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-3aaa437a-d9f5-4df3-895b-dc8d67aad0a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:48.005258 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:38:48.119354 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f29a3d-ecbf-4f3c-ae8b-bc73ca5e21af admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:48.121728 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 852/1709] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:38:47 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:38:48.122691 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaa437a-d9f5-4df3-895b-dc8d67aad0a1 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:48.134473 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-3aaa437a-d9f5-4df3-895b-dc8d67aad0a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:48.154640 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaa437a-d9f5-4df3-895b-dc8d67aad0a1 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:48.169007 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29759de-64fd-4ec9-ad17-153b72274d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:48.169671 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29759de-64fd-4ec9-ad17-153b72274d66 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:48.169988 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29759de-64fd-4ec9-ad17-153b72274d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:48.170231 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29759de-64fd-4ec9-ad17-153b72274d66 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:48.170710 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee594ec6-3471-4d17-bc11-0007dfd984ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:48.170967 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29759de-64fd-4ec9-ad17-153b72274d66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:48.172737 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29759de-64fd-4ec9-ad17-153b72274d66 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:48.173667 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee594ec6-3471-4d17-bc11-0007dfd984ef admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:48.175896 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee594ec6-3471-4d17-bc11-0007dfd984ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:48.176319 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee594ec6-3471-4d17-bc11-0007dfd984ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:48.177533 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee594ec6-3471-4d17-bc11-0007dfd984ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:48.180610 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee594ec6-3471-4d17-bc11-0007dfd984ef admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:48.195923 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 858/1710] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:38:47 2022] GET /identity/v3/roles => generated 1420 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:38:48.204471 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ecb4fc-0af3-4763-b68d-016873c85343 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:48.206211 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ecb4fc-0af3-4763-b68d-016873c85343 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:48.207039 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ecb4fc-0af3-4763-b68d-016873c85343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:48.214135 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 853/1711] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:38:48 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:38:48.214135 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29759de-64fd-4ec9-ad17-153b72274d66 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:48.220235 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-c29759de-64fd-4ec9-ad17-153b72274d66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:48.271836 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-85ecb4fc-0af3-4763-b68d-016873c85343 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:38:48.278050 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-85ecb4fc-0af3-4763-b68d-016873c85343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:48.299893 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 859/1712] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:38:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:38:48.311794 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ecb4fc-0af3-4763-b68d-016873c85343 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:48.343018 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ecb4fc-0af3-4763-b68d-016873c85343 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:48.355915 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e865e73-0a34-4a32-bd91-6e00046f1171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:48.357806 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e865e73-0a34-4a32-bd91-6e00046f1171 admin admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:48.358784 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e865e73-0a34-4a32-bd91-6e00046f1171 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:48.361099 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e865e73-0a34-4a32-bd91-6e00046f1171 admin admin] PATH_INFO: `/v3/projects/33cf9c15ba744a1fada8d251f1f4f02a/users/5e335978146e4224b1306fa52a17ff4e/roles/053e12af84e943e699460a9d76c2072b` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:48.406406 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29759de-64fd-4ec9-ad17-153b72274d66 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:48.421225 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8b7935-9ff4-47c5-8840-891ba0d7017e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a080c7ab9b684abca6828067f2572c3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:48.421882 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8b7935-9ff4-47c5-8840-891ba0d7017e service nova] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:48.422171 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8b7935-9ff4-47c5-8840-891ba0d7017e service nova] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:48.422459 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8b7935-9ff4-47c5-8840-891ba0d7017e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:48.439657 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e865e73-0a34-4a32-bd91-6e00046f1171 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33cf9c15ba744a1fada8d251f1f4f02a, user_id=5e335978146e4224b1306fa52a17ff4e, role_id=053e12af84e943e699460a9d76c2072b)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:48.454902 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8b7935-9ff4-47c5-8840-891ba0d7017e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:48.454902 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8b7935-9ff4-47c5-8840-891ba0d7017e service nova] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:48.455566 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e865e73-0a34-4a32-bd91-6e00046f1171 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:48.494722 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 854/1713] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:38:48 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:38:48.495604 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8b7935-9ff4-47c5-8840-891ba0d7017e service nova] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:48.521647 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8b7935-9ff4-47c5-8840-891ba0d7017e service nova] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:48.534281 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7e7acd-fad8-47be-afdf-e1c83c2f651f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:48.534970 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7e7acd-fad8-47be-afdf-e1c83c2f651f admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:48.535252 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7e7acd-fad8-47be-afdf-e1c83c2f651f admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:48.535536 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7e7acd-fad8-47be-afdf-e1c83c2f651f admin admin] PATH_INFO: `/v3/projects/c7ede89f19424467a1f33ad3b96449dc/users/c4c98774608f4be3a7a1be2a30d2ae33/roles/6bbb643e0d8d4bc0a9b27c26edf59216` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:48.566860 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7e7acd-fad8-47be-afdf-e1c83c2f651f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7ede89f19424467a1f33ad3b96449dc, user_id=c4c98774608f4be3a7a1be2a30d2ae33, role_id=6bbb643e0d8d4bc0a9b27c26edf59216)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:48.571051 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7e7acd-fad8-47be-afdf-e1c83c2f651f admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:48.574923 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 860/1714] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:38:48 2022] PUT /identity/v3/projects/33cf9c15ba744a1fada8d251f1f4f02a/users/5e335978146e4224b1306fa52a17ff4e/roles/053e12af84e943e699460a9d76c2072b => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:38:48.575632 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e865e73-0a34-4a32-bd91-6e00046f1171 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:48.582876 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-5e865e73-0a34-4a32-bd91-6e00046f1171 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:48.684724 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 855/1715] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:38:48 2022] PUT /identity/v3/projects/c7ede89f19424467a1f33ad3b96449dc/users/c4c98774608f4be3a7a1be2a30d2ae33/roles/6bbb643e0d8d4bc0a9b27c26edf59216 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:38:48.687430 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344868f5-a74b-4e22-942f-8f64a11877d3 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:48.687745 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344868f5-a74b-4e22-942f-8f64a11877d3 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:48.688101 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344868f5-a74b-4e22-942f-8f64a11877d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:48.704205 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e865e73-0a34-4a32-bd91-6e00046f1171 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:48.730066 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-344868f5-a74b-4e22-942f-8f64a11877d3 None None] MFA Rules not processed for user `5e335978146e4224b1306fa52a17ff4e`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:38:48.731448 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-344868f5-a74b-4e22-942f-8f64a11877d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:48.744138 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:38:48.812947 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604fe42b-da7c-4c5b-8227-524f142e1627 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:48.813828 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604fe42b-da7c-4c5b-8227-524f142e1627 service neutron] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:48.814339 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604fe42b-da7c-4c5b-8227-524f142e1627 service neutron] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:48.814793 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604fe42b-da7c-4c5b-8227-524f142e1627 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:48.842803 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 856/1716] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:38:48 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:38:48.844910 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344868f5-a74b-4e22-942f-8f64a11877d3 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:48.964624 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604fe42b-da7c-4c5b-8227-524f142e1627 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:48.971776 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604fe42b-da7c-4c5b-8227-524f142e1627 service neutron] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:48.990972 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344868f5-a74b-4e22-942f-8f64a11877d3 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:48.991743 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:38:49.008610 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 861/1717] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:38:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:38:49.008610 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604fe42b-da7c-4c5b-8227-524f142e1627 service neutron] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:49.011639 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0bdd76-8079-40e8-a089-77c970f1cad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:49.012672 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0bdd76-8079-40e8-a089-77c970f1cad5 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:49.012788 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0bdd76-8079-40e8-a089-77c970f1cad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:49.019009 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0bdd76-8079-40e8-a089-77c970f1cad5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:49.019888 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0bdd76-8079-40e8-a089-77c970f1cad5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:49.022047 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0bdd76-8079-40e8-a089-77c970f1cad5 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:49.038182 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604fe42b-da7c-4c5b-8227-524f142e1627 service neutron] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:49.048421 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 857/1718] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:38:48 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:38:49.052847 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0bdd76-8079-40e8-a089-77c970f1cad5 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:49.053880 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4612058d-c542-4d81-9046-5d2ea7bef7eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:49.054787 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4612058d-c542-4d81-9046-5d2ea7bef7eb admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:49.055097 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4612058d-c542-4d81-9046-5d2ea7bef7eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:49.055475 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4612058d-c542-4d81-9046-5d2ea7bef7eb admin admin] PATH_INFO: `/v3/projects` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:49.056524 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4612058d-c542-4d81-9046-5d2ea7bef7eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:49.058762 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4612058d-c542-4d81-9046-5d2ea7bef7eb admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:49.078011 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0bdd76-8079-40e8-a089-77c970f1cad5 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:49.083961 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cac32d-e823-4674-a801-47e37d819e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:49.084438 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cac32d-e823-4674-a801-47e37d819e1a admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:49.085679 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cac32d-e823-4674-a801-47e37d819e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:49.086220 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cac32d-e823-4674-a801-47e37d819e1a admin admin] PATH_INFO: `/v3/projects/c7ede89f19424467a1f33ad3b96449dc/users/c4c98774608f4be3a7a1be2a30d2ae33/roles/6bbb643e0d8d4bc0a9b27c26edf59216` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:49.088581 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cac32d-e823-4674-a801-47e37d819e1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7ede89f19424467a1f33ad3b96449dc, user_id=c4c98774608f4be3a7a1be2a30d2ae33, role_id=6bbb643e0d8d4bc0a9b27c26edf59216)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:49.091639 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cac32d-e823-4674-a801-47e37d819e1a admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:49.183318 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 858/1719] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:38:49 2022] PUT /identity/v3/projects/c7ede89f19424467a1f33ad3b96449dc/users/c4c98774608f4be3a7a1be2a30d2ae33/roles/6bbb643e0d8d4bc0a9b27c26edf59216 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:38:49.192268 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79989e91-2c62-4b76-8c72-d817cce47971 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:49.192733 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79989e91-2c62-4b76-8c72-d817cce47971 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:49.193022 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79989e91-2c62-4b76-8c72-d817cce47971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:49.226161 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-79989e91-2c62-4b76-8c72-d817cce47971 None None] MFA Rules not processed for user `c4c98774608f4be3a7a1be2a30d2ae33`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:38:49.227502 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-79989e91-2c62-4b76-8c72-d817cce47971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:49.359493 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 859/1720] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:38:49 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 07:38:49.366631 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79989e91-2c62-4b76-8c72-d817cce47971 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:49.368109 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-79989e91-2c62-4b76-8c72-d817cce47971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:49.454300 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79989e91-2c62-4b76-8c72-d817cce47971 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:49.457807 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 862/1721] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:38:49 2022] POST /identity/v3/projects => generated 393 bytes in 449 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:38:49.459927 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4612058d-c542-4d81-9046-5d2ea7bef7eb admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:49.581781 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec949ad-d879-4bdd-8819-8136126ca9a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecf6642b7b84d3c9756bb41ab3349d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:49.584292 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec949ad-d879-4bdd-8819-8136126ca9a8 service cinder] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:49.584694 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec949ad-d879-4bdd-8819-8136126ca9a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:49.585109 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec949ad-d879-4bdd-8819-8136126ca9a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:49.662513 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4612058d-c542-4d81-9046-5d2ea7bef7eb admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:49.671944 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd7c1ee-3aca-4d4c-a65d-e19ccb71791c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:49.673146 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd7c1ee-3aca-4d4c-a65d-e19ccb71791c admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:49.673802 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd7c1ee-3aca-4d4c-a65d-e19ccb71791c admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:49.674100 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd7c1ee-3aca-4d4c-a65d-e19ccb71791c admin admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:49.674748 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd7c1ee-3aca-4d4c-a65d-e19ccb71791c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:49.676705 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd7c1ee-3aca-4d4c-a65d-e19ccb71791c admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:49.800941 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 863/1722] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:38:49 2022] POST /identity/v3/users => generated 303 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:38:49.805904 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd7c1ee-3aca-4d4c-a65d-e19ccb71791c admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:49.830539 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:38:49.837689 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd7c1ee-3aca-4d4c-a65d-e19ccb71791c admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:49.871039 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6520e103-0d04-4209-88f4-8e06c1d3f2f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:49.871642 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6520e103-0d04-4209-88f4-8e06c1d3f2f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:49.871930 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6520e103-0d04-4209-88f4-8e06c1d3f2f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:49.875605 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6520e103-0d04-4209-88f4-8e06c1d3f2f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:49.876448 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6520e103-0d04-4209-88f4-8e06c1d3f2f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:49.897859 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6520e103-0d04-4209-88f4-8e06c1d3f2f0 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:49.898210 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec949ad-d879-4bdd-8819-8136126ca9a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:49.898210 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec949ad-d879-4bdd-8819-8136126ca9a8 service cinder] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:49.917226 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 864/1723] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:38:49 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:38:49.924023 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6520e103-0d04-4209-88f4-8e06c1d3f2f0 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:49.926479 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 860/1724] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:38:49 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 561 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 07:38:49.971684 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6520e103-0d04-4209-88f4-8e06c1d3f2f0 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:49.977085 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f61ffd-bf77-47ac-8b85-97f29db4a040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:49.977909 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f61ffd-bf77-47ac-8b85-97f29db4a040 admin admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:49.978197 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f61ffd-bf77-47ac-8b85-97f29db4a040 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:49.978482 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f61ffd-bf77-47ac-8b85-97f29db4a040 admin admin] PATH_INFO: `/v3/projects/3a481596310a44ef9667b1f960f75c02/users/c86981226dc940e589efaa310a53e6c5/roles/6bbb643e0d8d4bc0a9b27c26edf59216` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:49.999499 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f61ffd-bf77-47ac-8b85-97f29db4a040 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a481596310a44ef9667b1f960f75c02, user_id=c86981226dc940e589efaa310a53e6c5, role_id=6bbb643e0d8d4bc0a9b27c26edf59216)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:50.002918 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f61ffd-bf77-47ac-8b85-97f29db4a040 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:50.073610 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 865/1725] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:38:49 2022] PUT /identity/v3/projects/3a481596310a44ef9667b1f960f75c02/users/c86981226dc940e589efaa310a53e6c5/roles/6bbb643e0d8d4bc0a9b27c26edf59216 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:38:50.078917 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec949ad-d879-4bdd-8819-8136126ca9a8 service cinder] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:50.112475 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec949ad-d879-4bdd-8819-8136126ca9a8 service cinder] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:50.218247 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dcb16d-709c-4bfb-8c0d-f82aa0da0529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:50.219362 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dcb16d-709c-4bfb-8c0d-f82aa0da0529 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:50.219994 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dcb16d-709c-4bfb-8c0d-f82aa0da0529 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:50.220482 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dcb16d-709c-4bfb-8c0d-f82aa0da0529 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:50.221538 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dcb16d-709c-4bfb-8c0d-f82aa0da0529 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:50.223978 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dcb16d-709c-4bfb-8c0d-f82aa0da0529 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:50.245079 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 861/1726] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:38:50 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:38:50.249177 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f61ffd-bf77-47ac-8b85-97f29db4a040 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:50.278670 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f61ffd-bf77-47ac-8b85-97f29db4a040 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:50.288586 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0692f7a-fe98-40a2-91bb-6545bbacedc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:50.289491 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0692f7a-fe98-40a2-91bb-6545bbacedc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:50.289908 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0692f7a-fe98-40a2-91bb-6545bbacedc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:50.290186 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0692f7a-fe98-40a2-91bb-6545bbacedc5 admin admin] PATH_INFO: `/v3/projects/3a481596310a44ef9667b1f960f75c02/users/c86981226dc940e589efaa310a53e6c5/roles/6bbb643e0d8d4bc0a9b27c26edf59216` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:50.293466 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0692f7a-fe98-40a2-91bb-6545bbacedc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a481596310a44ef9667b1f960f75c02, user_id=c86981226dc940e589efaa310a53e6c5, role_id=6bbb643e0d8d4bc0a9b27c26edf59216)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:50.300195 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0692f7a-fe98-40a2-91bb-6545bbacedc5 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:50.334658 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 866/1727] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:38:50 2022] PUT /identity/v3/projects/3a481596310a44ef9667b1f960f75c02/users/c86981226dc940e589efaa310a53e6c5/roles/6bbb643e0d8d4bc0a9b27c26edf59216 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:38:50.337540 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a9ce7f-4ce2-4cbc-881a-2fe16514600f None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:50.337540 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a9ce7f-4ce2-4cbc-881a-2fe16514600f None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:50.337540 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a9ce7f-4ce2-4cbc-881a-2fe16514600f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:50.392405 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-b5a9ce7f-4ce2-4cbc-881a-2fe16514600f None None] MFA Rules not processed for user `c86981226dc940e589efaa310a53e6c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:38:50.409751 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-b5a9ce7f-4ce2-4cbc-881a-2fe16514600f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:50.491736 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0692f7a-fe98-40a2-91bb-6545bbacedc5 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:50.494911 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-c0692f7a-fe98-40a2-91bb-6545bbacedc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:50.522697 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:38:50.639959 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0692f7a-fe98-40a2-91bb-6545bbacedc5 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:50.644212 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7f2d06-f1b1-421f-a625-a0ba993cf4c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecf6642b7b84d3c9756bb41ab3349d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:50.645409 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7f2d06-f1b1-421f-a625-a0ba993cf4c7 service cinder] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:50.651343 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7f2d06-f1b1-421f-a625-a0ba993cf4c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:50.651801 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7f2d06-f1b1-421f-a625-a0ba993cf4c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:50.661492 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-dd7f2d06-f1b1-421f-a625-a0ba993cf4c7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:50.664209 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 862/1728] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:38:50 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:38:50.754963 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a9ce7f-4ce2-4cbc-881a-2fe16514600f None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:50.766870 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-b5a9ce7f-4ce2-4cbc-881a-2fe16514600f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:50.802658 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:38:50.884520 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7f2d06-f1b1-421f-a625-a0ba993cf4c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:50.886523 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7f2d06-f1b1-421f-a625-a0ba993cf4c7 service cinder] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:50.933811 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 867/1729] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:38:50 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 07:38:50.987886 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a9ce7f-4ce2-4cbc-881a-2fe16514600f None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:51.007193 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97c1999-799c-4301-b9b7-9b8a3acb5e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:51.008227 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97c1999-799c-4301-b9b7-9b8a3acb5e37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:51.008537 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97c1999-799c-4301-b9b7-9b8a3acb5e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:51.008835 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97c1999-799c-4301-b9b7-9b8a3acb5e37 admin admin] PATH_INFO: `/v3/projects/bc55a1b8894a43bdaeef8a3ae59bd1dc` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:51.013363 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97c1999-799c-4301-b9b7-9b8a3acb5e37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc55a1b8894a43bdaeef8a3ae59bd1dc)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:51.015422 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97c1999-799c-4301-b9b7-9b8a3acb5e37 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:51.075766 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e63cd91-db48-49ab-89cd-cbe9fe107883 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:51.076328 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e63cd91-db48-49ab-89cd-cbe9fe107883 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:51.076516 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e63cd91-db48-49ab-89cd-cbe9fe107883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:51.228664 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-3e63cd91-db48-49ab-89cd-cbe9fe107883 None None] MFA Rules not processed for user `f9c68c4bd9074d8d9b893755e28c0807`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:38:51.231900 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-3e63cd91-db48-49ab-89cd-cbe9fe107883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:51.238791 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-f97c1999-799c-4301-b9b7-9b8a3acb5e37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc55a1b8894a43bdaeef8a3ae59bd1dc', 'request_id': 'req-f97c1999-799c-4301-b9b7-9b8a3acb5e37'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:38:51.255582 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-f97c1999-799c-4301-b9b7-9b8a3acb5e37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc55a1b8894a43bdaeef8a3ae59bd1dc', 'request_id': 'req-f97c1999-799c-4301-b9b7-9b8a3acb5e37'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:38:51.342282 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-f97c1999-799c-4301-b9b7-9b8a3acb5e37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc55a1b8894a43bdaeef8a3ae59bd1dc', 'request_id': 'req-f97c1999-799c-4301-b9b7-9b8a3acb5e37'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:38:51.343986 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 868/1730] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Sep 15 07:38:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 07:38:51.344179 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e63cd91-db48-49ab-89cd-cbe9fe107883 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:51.367831 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-f97c1999-799c-4301-b9b7-9b8a3acb5e37 admin admin] The token cache is being invalidate because project bc55a1b8894a43bdaeef8a3ae59bd1dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:38:51.368134 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-f97c1999-799c-4301-b9b7-9b8a3acb5e37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:38:51.371567 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-f97c1999-799c-4301-b9b7-9b8a3acb5e37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:38:51.375528 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 863/1731] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:38:50 2022] DELETE /identity/v3/projects/bc55a1b8894a43bdaeef8a3ae59bd1dc => generated 0 bytes in 621 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 07:38:51.382827 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421d8188-52c3-4903-b6a6-c727494c368f None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:51.383204 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421d8188-52c3-4903-b6a6-c727494c368f None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:51.383492 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421d8188-52c3-4903-b6a6-c727494c368f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:51.430234 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e63cd91-db48-49ab-89cd-cbe9fe107883 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:51.430234 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e68486-4034-43df-a779-1d337737ac36 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecf6642b7b84d3c9756bb41ab3349d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:51.430234 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e68486-4034-43df-a779-1d337737ac36 service cinder] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:51.430234 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e68486-4034-43df-a779-1d337737ac36 service cinder] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:51.430694 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e68486-4034-43df-a779-1d337737ac36 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:51.433070 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-41e68486-4034-43df-a779-1d337737ac36 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:51.434719 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-421d8188-52c3-4903-b6a6-c727494c368f None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:38:51.435927 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-421d8188-52c3-4903-b6a6-c727494c368f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:51.502274 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:38:51.546921 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:38:51.560966 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e68486-4034-43df-a779-1d337737ac36 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:51.565436 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e68486-4034-43df-a779-1d337737ac36 service cinder] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:51.574521 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 864/1732] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:38:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:38:51.576227 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4268446f-5ef6-424e-89de-7e5ac2635e96 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:51.576521 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4268446f-5ef6-424e-89de-7e5ac2635e96 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:51.576799 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4268446f-5ef6-424e-89de-7e5ac2635e96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:51.595697 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 869/1733] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:38:51 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:38:51.622216 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-4268446f-5ef6-424e-89de-7e5ac2635e96 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:38:51.623498 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-4268446f-5ef6-424e-89de-7e5ac2635e96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:51.641460 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 865/1734] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:38:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 07:38:51.645735 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e68486-4034-43df-a779-1d337737ac36 service cinder] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:51.681174 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e68486-4034-43df-a779-1d337737ac36 service cinder] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:51.702619 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f08b6a-1a2c-419f-a54d-1141136d2721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:51.705010 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f08b6a-1a2c-419f-a54d-1141136d2721 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:51.709097 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f08b6a-1a2c-419f-a54d-1141136d2721 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:51.709097 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f08b6a-1a2c-419f-a54d-1141136d2721 admin admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:51.709097 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f08b6a-1a2c-419f-a54d-1141136d2721 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:38:51.715446 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f08b6a-1a2c-419f-a54d-1141136d2721 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:51.722473 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f08b6a-1a2c-419f-a54d-1141136d2721 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:51.814243 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 870/1735] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 15 07:38:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:38:51.817726 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4268446f-5ef6-424e-89de-7e5ac2635e96 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:51.830603 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f08b6a-1a2c-419f-a54d-1141136d2721 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:51.831530 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4268446f-5ef6-424e-89de-7e5ac2635e96 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:51.834628 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-c9f08b6a-1a2c-419f-a54d-1141136d2721 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:51.854769 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f08b6a-1a2c-419f-a54d-1141136d2721 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:51.857427 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45dcb40-36f5-477f-b837-471c755ac2af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:51.858165 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45dcb40-36f5-477f-b837-471c755ac2af admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:51.860016 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45dcb40-36f5-477f-b837-471c755ac2af admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:51.860016 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45dcb40-36f5-477f-b837-471c755ac2af admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:51.860016 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45dcb40-36f5-477f-b837-471c755ac2af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:51.861291 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45dcb40-36f5-477f-b837-471c755ac2af admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:51.863778 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e59da8-fd85-47cc-93a1-ca8bb3098fd5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecf6642b7b84d3c9756bb41ab3349d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:51.864540 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e59da8-fd85-47cc-93a1-ca8bb3098fd5 service cinder] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:51.864971 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e59da8-fd85-47cc-93a1-ca8bb3098fd5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:51.865356 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e59da8-fd85-47cc-93a1-ca8bb3098fd5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:51.868130 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-f1e59da8-fd85-47cc-93a1-ca8bb3098fd5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:52.025341 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 866/1736] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:38:51 2022] POST /identity/v3/projects => generated 405 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:38:52.026431 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e59da8-fd85-47cc-93a1-ca8bb3098fd5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:52.026555 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45dcb40-36f5-477f-b837-471c755ac2af admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:52.027349 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e59da8-fd85-47cc-93a1-ca8bb3098fd5 service cinder] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:52.028133 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-b45dcb40-36f5-477f-b837-471c755ac2af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:52.044882 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 871/1737] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:38:51 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 07:38:52.046667 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e59da8-fd85-47cc-93a1-ca8bb3098fd5 service cinder] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:52.061228 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e59da8-fd85-47cc-93a1-ca8bb3098fd5 service cinder] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:52.182193 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27826e30-8505-407f-ae4e-9468df12e462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:52.187642 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27826e30-8505-407f-ae4e-9468df12e462 admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:52.188793 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27826e30-8505-407f-ae4e-9468df12e462 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:52.189209 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27826e30-8505-407f-ae4e-9468df12e462 admin admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:52.193633 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27826e30-8505-407f-ae4e-9468df12e462 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:52.199093 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27826e30-8505-407f-ae4e-9468df12e462 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:52.265986 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45dcb40-36f5-477f-b837-471c755ac2af admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:52.279387 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ede586-87f3-4743-9cca-c7bd9a033ec5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72b90d35087e4d7a80e9417082a1541f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:52.279387 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ede586-87f3-4743-9cca-c7bd9a033ec5 service swift] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:52.279387 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ede586-87f3-4743-9cca-c7bd9a033ec5 service swift] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:52.279387 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ede586-87f3-4743-9cca-c7bd9a033ec5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:52.281733 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-94ede586-87f3-4743-9cca-c7bd9a033ec5 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:52.364785 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 872/1738] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:38:52 2022] POST /identity/v3/users => generated 303 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:38:52.366610 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27826e30-8505-407f-ae4e-9468df12e462 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:52.369264 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ede586-87f3-4743-9cca-c7bd9a033ec5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:52.371347 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ede586-87f3-4743-9cca-c7bd9a033ec5 service swift] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:52.384147 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27826e30-8505-407f-ae4e-9468df12e462 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:52.398461 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fba3bd-431d-468f-beda-b6278b5ac1e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:52.399026 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fba3bd-431d-468f-beda-b6278b5ac1e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:52.399272 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fba3bd-431d-468f-beda-b6278b5ac1e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:52.399553 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fba3bd-431d-468f-beda-b6278b5ac1e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:52.400271 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fba3bd-431d-468f-beda-b6278b5ac1e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:52.403787 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fba3bd-431d-468f-beda-b6278b5ac1e2 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:52.411915 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 867/1739] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Sep 15 07:38:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 07:38:52.459325 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 873/1740] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:38:52 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 07:38:52.461243 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ede586-87f3-4743-9cca-c7bd9a033ec5 service swift] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:52.477331 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ede586-87f3-4743-9cca-c7bd9a033ec5 service swift] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:52.499225 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519859d7-3bd5-4b32-9be3-9385fb01887d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:52.499883 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519859d7-3bd5-4b32-9be3-9385fb01887d admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:52.500251 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519859d7-3bd5-4b32-9be3-9385fb01887d admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:52.500533 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519859d7-3bd5-4b32-9be3-9385fb01887d admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:52.501195 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519859d7-3bd5-4b32-9be3-9385fb01887d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:52.503956 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519859d7-3bd5-4b32-9be3-9385fb01887d admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:52.536329 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 868/1741] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:38:52 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:38:52.538432 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fba3bd-431d-468f-beda-b6278b5ac1e2 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:52.589941 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fba3bd-431d-468f-beda-b6278b5ac1e2 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:52.617991 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a99ab2-2259-4a60-a53d-91bb9204d292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:52.619671 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a99ab2-2259-4a60-a53d-91bb9204d292 admin admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:52.620096 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a99ab2-2259-4a60-a53d-91bb9204d292 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:52.620485 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a99ab2-2259-4a60-a53d-91bb9204d292 admin admin] PATH_INFO: `/v3/projects/3254ca4f60ec4def824456f9887dde02/users/056886a1210144f1ac838ac94b6eac6e/roles/053e12af84e943e699460a9d76c2072b` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:52.650830 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a99ab2-2259-4a60-a53d-91bb9204d292 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3254ca4f60ec4def824456f9887dde02, user_id=056886a1210144f1ac838ac94b6eac6e, role_id=053e12af84e943e699460a9d76c2072b)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:52.655839 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a99ab2-2259-4a60-a53d-91bb9204d292 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:52.761299 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2253dad-4d93-4658-a912-5ac5cd1425e9 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:52.761737 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2253dad-4d93-4658-a912-5ac5cd1425e9 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:52.762075 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2253dad-4d93-4658-a912-5ac5cd1425e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:52.768534 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 874/1742] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:38:52 2022] PUT /identity/v3/projects/3254ca4f60ec4def824456f9887dde02/users/056886a1210144f1ac838ac94b6eac6e/roles/053e12af84e943e699460a9d76c2072b => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 07:38:52.770580 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a2410a-3d02-4198-8211-b94636e5421f None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:52.771077 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a2410a-3d02-4198-8211-b94636e5421f None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:52.771737 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a2410a-3d02-4198-8211-b94636e5421f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:52.850155 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-39a2410a-3d02-4198-8211-b94636e5421f None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:38:52.851539 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-39a2410a-3d02-4198-8211-b94636e5421f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:52.860497 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-c2253dad-4d93-4658-a912-5ac5cd1425e9 None None] MFA Rules not processed for user `5315996432f44bffa3460b2744c10f0a`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:38:52.862367 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-c2253dad-4d93-4658-a912-5ac5cd1425e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:53.083058 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:38:53.093874 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:38:53.117893 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 869/1743] 10.222.0.26 () {58 vars in 1015 bytes} [Thu Sep 15 07:38:52 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 357 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:38:53.164125 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2253dad-4d93-4658-a912-5ac5cd1425e9 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:53.165979 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-c2253dad-4d93-4658-a912-5ac5cd1425e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:53.183288 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2253dad-4d93-4658-a912-5ac5cd1425e9 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:53.201431 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed8bda0-62a7-4e49-9a88-2f0c001e830c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:53.202100 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed8bda0-62a7-4e49-9a88-2f0c001e830c service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:53.202387 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed8bda0-62a7-4e49-9a88-2f0c001e830c service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:53.216990 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed8bda0-62a7-4e49-9a88-2f0c001e830c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:53.217920 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 875/1744] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:38:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 448 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:38:53.237601 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed8bda0-62a7-4e49-9a88-2f0c001e830c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:53.250163 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed8bda0-62a7-4e49-9a88-2f0c001e830c service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:53.278053 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 870/1745] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:38:53 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 07:38:53.321933 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a2410a-3d02-4198-8211-b94636e5421f None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:53.354261 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a2410a-3d02-4198-8211-b94636e5421f None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:53.364443 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fc8460-ef78-4a2f-a8a2-e7293a836968 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:53.365188 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fc8460-ef78-4a2f-a8a2-e7293a836968 service neutron] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:53.365417 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fc8460-ef78-4a2f-a8a2-e7293a836968 service neutron] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:53.365736 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fc8460-ef78-4a2f-a8a2-e7293a836968 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:53.386518 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fc8460-ef78-4a2f-a8a2-e7293a836968 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:53.391266 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fc8460-ef78-4a2f-a8a2-e7293a836968 service neutron] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:53.431800 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 876/1746] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:38:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:38:54.090479 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed8bda0-62a7-4e49-9a88-2f0c001e830c service neutron] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:54.092607 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-eed8bda0-62a7-4e49-9a88-2f0c001e830c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:54.129826 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed8bda0-62a7-4e49-9a88-2f0c001e830c service neutron] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:54.146142 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b0d65d-0961-4b67-8486-bccfcf635554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:54.147966 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b0d65d-0961-4b67-8486-bccfcf635554 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:54.148443 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b0d65d-0961-4b67-8486-bccfcf635554 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:54.148892 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b0d65d-0961-4b67-8486-bccfcf635554 admin admin] PATH_INFO: `/v3/users/96bbf934e49b4bff9810a8e11c55b49b` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:54.153173 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b0d65d-0961-4b67-8486-bccfcf635554 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96bbf934e49b4bff9810a8e11c55b49b)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:54.155356 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b0d65d-0961-4b67-8486-bccfcf635554 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:54.452973 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435323c9-f381-4f25-9b3e-e8a500792fb4 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:54.453512 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435323c9-f381-4f25-9b3e-e8a500792fb4 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:54.453942 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435323c9-f381-4f25-9b3e-e8a500792fb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:54.512417 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-e9b0d65d-0961-4b67-8486-bccfcf635554 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96bbf934e49b4bff9810a8e11c55b49b'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:38:54.513620 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-e9b0d65d-0961-4b67-8486-bccfcf635554 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96bbf934e49b4bff9810a8e11c55b49b'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:38:54.520390 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-435323c9-f381-4f25-9b3e-e8a500792fb4 None None] MFA Rules not processed for user `5e0c8d7aa4db455a94631becf212866f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:38:54.521608 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-435323c9-f381-4f25-9b3e-e8a500792fb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:54.717145 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-e9b0d65d-0961-4b67-8486-bccfcf635554 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96bbf934e49b4bff9810a8e11c55b49b'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:38:54.718082 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-e9b0d65d-0961-4b67-8486-bccfcf635554 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96bbf934e49b4bff9810a8e11c55b49b'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:38:54.731056 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-e9b0d65d-0961-4b67-8486-bccfcf635554 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96bbf934e49b4bff9810a8e11c55b49b'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:38:54.739768 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 877/1747] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:38:54 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:38:54.802714 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 871/1748] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:38:54 2022] DELETE /identity/v3/users/96bbf934e49b4bff9810a8e11c55b49b => generated 0 bytes in 713 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:38:54.815655 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f334ea6-391b-46b1-87db-612261d76e01 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:54.815655 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f334ea6-391b-46b1-87db-612261d76e01 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:54.815655 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f334ea6-391b-46b1-87db-612261d76e01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:54.902163 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-7f334ea6-391b-46b1-87db-612261d76e01 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:38:54.923628 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-7f334ea6-391b-46b1-87db-612261d76e01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:55.126470 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 878/1749] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:38:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:38:55.146684 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b0d65d-0961-4b67-8486-bccfcf635554 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:55.146684 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-e9b0d65d-0961-4b67-8486-bccfcf635554 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:55.282339 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b0d65d-0961-4b67-8486-bccfcf635554 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:55.286618 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350c1b0c-1816-466b-876f-5f49a735b0f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:55.287485 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350c1b0c-1816-466b-876f-5f49a735b0f9 service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:55.287955 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350c1b0c-1816-466b-876f-5f49a735b0f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:55.288456 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350c1b0c-1816-466b-876f-5f49a735b0f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:55.303077 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350c1b0c-1816-466b-876f-5f49a735b0f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:55.305718 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350c1b0c-1816-466b-876f-5f49a735b0f9 service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:55.328275 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 872/1750] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:38:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:38:55.814373 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3b6915-bf12-466b-9847-2bbdd7e287c0 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:55.814935 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3b6915-bf12-466b-9847-2bbdd7e287c0 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:55.815292 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3b6915-bf12-466b-9847-2bbdd7e287c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:55.871936 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-ae3b6915-bf12-466b-9847-2bbdd7e287c0 None None] MFA Rules not processed for user `56c2f854cf074b1ba2571766b7f2fc51`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:38:55.876921 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-ae3b6915-bf12-466b-9847-2bbdd7e287c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:56.068967 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 879/1751] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:38:55 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:38:56.224058 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350c1b0c-1816-466b-876f-5f49a735b0f9 service neutron] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:56.227297 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-350c1b0c-1816-466b-876f-5f49a735b0f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:56.311579 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350c1b0c-1816-466b-876f-5f49a735b0f9 service neutron] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:56.345257 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68205189-8438-42bc-a876-2ec59ff5eeeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:56.345954 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68205189-8438-42bc-a876-2ec59ff5eeeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:56.346239 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68205189-8438-42bc-a876-2ec59ff5eeeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:56.346515 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68205189-8438-42bc-a876-2ec59ff5eeeb admin admin] PATH_INFO: `/v3/projects/166bf7277833431ebc3e8abc17f10c0d` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:56.355315 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68205189-8438-42bc-a876-2ec59ff5eeeb admin admin] RBAC: Authorizing `identity:delete_project(project_id=166bf7277833431ebc3e8abc17f10c0d)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:56.357260 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68205189-8438-42bc-a876-2ec59ff5eeeb admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:56.475169 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:38:56.596633 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-68205189-8438-42bc-a876-2ec59ff5eeeb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '166bf7277833431ebc3e8abc17f10c0d', 'request_id': 'req-68205189-8438-42bc-a876-2ec59ff5eeeb'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:38:56.604981 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-68205189-8438-42bc-a876-2ec59ff5eeeb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '166bf7277833431ebc3e8abc17f10c0d', 'request_id': 'req-68205189-8438-42bc-a876-2ec59ff5eeeb'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:38:56.721992 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-68205189-8438-42bc-a876-2ec59ff5eeeb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '166bf7277833431ebc3e8abc17f10c0d', 'request_id': 'req-68205189-8438-42bc-a876-2ec59ff5eeeb'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:38:56.753342 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-68205189-8438-42bc-a876-2ec59ff5eeeb admin admin] The token cache is being invalidate because project 166bf7277833431ebc3e8abc17f10c0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:38:56.753921 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-68205189-8438-42bc-a876-2ec59ff5eeeb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:38:56.755125 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cabc185-9c76-4be7-8701-7480f3abe39d None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:56.755549 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cabc185-9c76-4be7-8701-7480f3abe39d None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:56.755826 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cabc185-9c76-4be7-8701-7480f3abe39d None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:56.756275 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-68205189-8438-42bc-a876-2ec59ff5eeeb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:38:56.758962 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 880/1752] 10.222.0.42 () {54 vars in 912 bytes} [Thu Sep 15 07:38:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 07:38:56.764922 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 873/1753] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:38:56 2022] DELETE /identity/v3/projects/166bf7277833431ebc3e8abc17f10c0d => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:38:56.765203 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d0a737-07a1-4e42-bb8c-969abd69abfd None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:56.765645 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d0a737-07a1-4e42-bb8c-969abd69abfd None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:56.765838 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d0a737-07a1-4e42-bb8c-969abd69abfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:56.771080 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68205189-8438-42bc-a876-2ec59ff5eeeb admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:56.776870 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-68205189-8438-42bc-a876-2ec59ff5eeeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:56.790966 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-d8d0a737-07a1-4e42-bb8c-969abd69abfd None None] MFA Rules not processed for user `5315996432f44bffa3460b2744c10f0a`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:38:56.792553 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-d8d0a737-07a1-4e42-bb8c-969abd69abfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:56.826562 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:38:56.916119 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 881/1754] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Sep 15 07:38:56 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 07:38:56.934900 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68205189-8438-42bc-a876-2ec59ff5eeeb admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:56.942301 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bd34ae-6730-43bd-a01d-4ac6a7163858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:56.943716 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd34ae-6730-43bd-a01d-4ac6a7163858 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:56.945104 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd34ae-6730-43bd-a01d-4ac6a7163858 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:56.945648 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd34ae-6730-43bd-a01d-4ac6a7163858 admin admin] PATH_INFO: `/v3/users/1e4bddfa72b44a37adabc00a1099944f` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:56.949507 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bd34ae-6730-43bd-a01d-4ac6a7163858 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e4bddfa72b44a37adabc00a1099944f)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:56.954839 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bd34ae-6730-43bd-a01d-4ac6a7163858 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:56.976104 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d0a737-07a1-4e42-bb8c-969abd69abfd None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:56.982461 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-d8d0a737-07a1-4e42-bb8c-969abd69abfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:57.017824 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d0a737-07a1-4e42-bb8c-969abd69abfd None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:57.029695 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe34769-1af9-4fe3-a092-b3147ce60be7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:57.030351 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe34769-1af9-4fe3-a092-b3147ce60be7 service neutron] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:57.030755 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe34769-1af9-4fe3-a092-b3147ce60be7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:57.031034 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe34769-1af9-4fe3-a092-b3147ce60be7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:57.056172 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe34769-1af9-4fe3-a092-b3147ce60be7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:57.057693 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe34769-1af9-4fe3-a092-b3147ce60be7 service neutron] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:57.078789 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 882/1755] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:38:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 07:38:57.290341 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b85b01-8663-447b-bcb2-5c4eb3aa976a None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:57.290653 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b85b01-8663-447b-bcb2-5c4eb3aa976a None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:57.291043 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b85b01-8663-447b-bcb2-5c4eb3aa976a None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:57.293261 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 883/1756] 10.222.0.42 () {54 vars in 921 bytes} [Thu Sep 15 07:38:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:38:57.333325 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00cab53-46dc-4b6d-b517-5bdf7f583f7d None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:57.333524 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00cab53-46dc-4b6d-b517-5bdf7f583f7d None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:57.333845 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00cab53-46dc-4b6d-b517-5bdf7f583f7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:57.339708 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-60bd34ae-6730-43bd-a01d-4ac6a7163858 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e4bddfa72b44a37adabc00a1099944f'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:38:57.341359 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-60bd34ae-6730-43bd-a01d-4ac6a7163858 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e4bddfa72b44a37adabc00a1099944f'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:38:57.433897 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-f00cab53-46dc-4b6d-b517-5bdf7f583f7d None None] MFA Rules not processed for user `a080c7ab9b684abca6828067f2572c3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:38:57.436101 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-f00cab53-46dc-4b6d-b517-5bdf7f583f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:57.448637 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-60bd34ae-6730-43bd-a01d-4ac6a7163858 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e4bddfa72b44a37adabc00a1099944f'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:38:57.449424 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-60bd34ae-6730-43bd-a01d-4ac6a7163858 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e4bddfa72b44a37adabc00a1099944f'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:38:57.464922 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-60bd34ae-6730-43bd-a01d-4ac6a7163858 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e4bddfa72b44a37adabc00a1099944f'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:38:57.527756 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 874/1757] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:38:56 2022] DELETE /identity/v3/users/1e4bddfa72b44a37adabc00a1099944f => generated 0 bytes in 757 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:38:57.715779 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 884/1758] 10.222.0.42 () {58 vars in 1018 bytes} [Thu Sep 15 07:38:57 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 389 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:38:58.555839 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5a9aad-6cf3-4694-baf3-b2d01acda56c None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:58.557038 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5a9aad-6cf3-4694-baf3-b2d01acda56c None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:58.557528 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5a9aad-6cf3-4694-baf3-b2d01acda56c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:58.652831 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-7f5a9aad-6cf3-4694-baf3-b2d01acda56c None None] MFA Rules not processed for user `056886a1210144f1ac838ac94b6eac6e`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:38:58.656169 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-7f5a9aad-6cf3-4694-baf3-b2d01acda56c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:58.685834 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:38:58.691666 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00cab53-46dc-4b6d-b517-5bdf7f583f7d None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:58.692744 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-f00cab53-46dc-4b6d-b517-5bdf7f583f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:58.737929 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00cab53-46dc-4b6d-b517-5bdf7f583f7d None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:58.800005 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 875/1759] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:38:58 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:38:58.817936 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5a9aad-6cf3-4694-baf3-b2d01acda56c None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:58.820744 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-7f5a9aad-6cf3-4694-baf3-b2d01acda56c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:58.960716 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13c5ff1-f852-4086-873a-e190e17c0cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:58.960716 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13c5ff1-f852-4086-873a-e190e17c0cb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:58.960716 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13c5ff1-f852-4086-873a-e190e17c0cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:58.960716 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13c5ff1-f852-4086-873a-e190e17c0cb2 admin admin] PATH_INFO: `/v3/projects/b6a4ddb789f04146b628a5ef7e5d4d4d` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:58.967093 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13c5ff1-f852-4086-873a-e190e17c0cb2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6a4ddb789f04146b628a5ef7e5d4d4d)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:58.973268 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13c5ff1-f852-4086-873a-e190e17c0cb2 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:59.070186 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5a9aad-6cf3-4694-baf3-b2d01acda56c None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:59.081195 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a564610-a9d7-4e9b-b688-3d5c2f1cd4c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:59.082403 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a564610-a9d7-4e9b-b688-3d5c2f1cd4c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:59.083023 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a564610-a9d7-4e9b-b688-3d5c2f1cd4c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:59.083443 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a564610-a9d7-4e9b-b688-3d5c2f1cd4c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:59.084260 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a564610-a9d7-4e9b-b688-3d5c2f1cd4c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:59.095626 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a564610-a9d7-4e9b-b688-3d5c2f1cd4c8 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:59.145168 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:38:59.153558 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-c13c5ff1-f852-4086-873a-e190e17c0cb2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6a4ddb789f04146b628a5ef7e5d4d4d', 'request_id': 'req-c13c5ff1-f852-4086-873a-e190e17c0cb2'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:38:59.169376 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-c13c5ff1-f852-4086-873a-e190e17c0cb2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6a4ddb789f04146b628a5ef7e5d4d4d', 'request_id': 'req-c13c5ff1-f852-4086-873a-e190e17c0cb2'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:38:59.225313 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 876/1760] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:38:58 2022] POST /identity/v3/projects => generated 403 bytes in 422 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 07:38:59.229000 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a564610-a9d7-4e9b-b688-3d5c2f1cd4c8 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:59.231255 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-2a564610-a9d7-4e9b-b688-3d5c2f1cd4c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:59.317423 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-c13c5ff1-f852-4086-873a-e190e17c0cb2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6a4ddb789f04146b628a5ef7e5d4d4d', 'request_id': 'req-c13c5ff1-f852-4086-873a-e190e17c0cb2'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:38:59.340584 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-c13c5ff1-f852-4086-873a-e190e17c0cb2 admin admin] The token cache is being invalidate because project b6a4ddb789f04146b628a5ef7e5d4d4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:38:59.341102 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-c13c5ff1-f852-4086-873a-e190e17c0cb2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:38:59.344261 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-c13c5ff1-f852-4086-873a-e190e17c0cb2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:38:59.346038 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 885/1761] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:38:58 2022] DELETE /identity/v3/projects/b6a4ddb789f04146b628a5ef7e5d4d4d => generated 0 bytes in 656 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 07:38:59.347041 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13c5ff1-f852-4086-873a-e190e17c0cb2 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:59.348646 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-c13c5ff1-f852-4086-873a-e190e17c0cb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:59.419974 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a564610-a9d7-4e9b-b688-3d5c2f1cd4c8 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:59.440285 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a386dda-cc3a-4327-b9c8-e7f0b92b0099 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a080c7ab9b684abca6828067f2572c3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:59.440285 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a386dda-cc3a-4327-b9c8-e7f0b92b0099 service nova] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:59.440285 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a386dda-cc3a-4327-b9c8-e7f0b92b0099 service nova] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:59.440285 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a386dda-cc3a-4327-b9c8-e7f0b92b0099 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:59.453038 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-3a386dda-cc3a-4327-b9c8-e7f0b92b0099 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:59.485597 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a386dda-cc3a-4327-b9c8-e7f0b92b0099 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:59.487157 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a386dda-cc3a-4327-b9c8-e7f0b92b0099 service nova] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:59.497981 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13c5ff1-f852-4086-873a-e190e17c0cb2 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:59.514542 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 877/1762] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:38:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:38:59.517945 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d7ae22-8227-46e9-a627-7dcb8b7ffc02 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:59.517945 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d7ae22-8227-46e9-a627-7dcb8b7ffc02 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:59.517945 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d7ae22-8227-46e9-a627-7dcb8b7ffc02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:59.541797 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-35d7ae22-8227-46e9-a627-7dcb8b7ffc02 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:38:59.546574 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-35d7ae22-8227-46e9-a627-7dcb8b7ffc02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:59.635837 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b960304e-8c4e-45fa-bffa-cb8282f60823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:59.652941 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b960304e-8c4e-45fa-bffa-cb8282f60823 admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:59.652941 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b960304e-8c4e-45fa-bffa-cb8282f60823 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:59.652941 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b960304e-8c4e-45fa-bffa-cb8282f60823 admin admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:59.652941 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b960304e-8c4e-45fa-bffa-cb8282f60823 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:59.652941 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b960304e-8c4e-45fa-bffa-cb8282f60823 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:59.684821 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 878/1763] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:38:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:38:59.686145 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d7ae22-8227-46e9-a627-7dcb8b7ffc02 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:38:59.687714 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-35d7ae22-8227-46e9-a627-7dcb8b7ffc02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:38:59.704897 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d7ae22-8227-46e9-a627-7dcb8b7ffc02 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:38:59.714054 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587df126-4066-48e4-a977-3ca304528e44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:38:59.715220 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587df126-4066-48e4-a977-3ca304528e44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:59.715824 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587df126-4066-48e4-a977-3ca304528e44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:59.716243 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587df126-4066-48e4-a977-3ca304528e44 admin admin] PATH_INFO: `/v3/users/5e335978146e4224b1306fa52a17ff4e` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:38:59.718034 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587df126-4066-48e4-a977-3ca304528e44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e335978146e4224b1306fa52a17ff4e)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:38:59.720558 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587df126-4066-48e4-a977-3ca304528e44 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:38:59.976297 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 886/1764] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:38:59 2022] POST /identity/v3/users => generated 308 bytes in 629 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:38:59.977558 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be57f3a-1160-4927-95d8-ba5551021d1a None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:38:59.977871 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be57f3a-1160-4927-95d8-ba5551021d1a None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:38:59.978210 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be57f3a-1160-4927-95d8-ba5551021d1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:00.014411 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-1be57f3a-1160-4927-95d8-ba5551021d1a None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:39:00.015757 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-1be57f3a-1160-4927-95d8-ba5551021d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:00.029452 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 887/1765] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:38:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:39:00.029452 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be57f3a-1160-4927-95d8-ba5551021d1a None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:00.050494 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be57f3a-1160-4927-95d8-ba5551021d1a None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:00.053153 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93026849-7b42-45a2-9e51-b6167099c303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:00.054237 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93026849-7b42-45a2-9e51-b6167099c303 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:00.054783 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93026849-7b42-45a2-9e51-b6167099c303 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:00.055282 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93026849-7b42-45a2-9e51-b6167099c303 admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:00.056311 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93026849-7b42-45a2-9e51-b6167099c303 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:00.058523 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93026849-7b42-45a2-9e51-b6167099c303 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:00.068891 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-587df126-4066-48e4-a977-3ca304528e44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e335978146e4224b1306fa52a17ff4e'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:00.071073 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-587df126-4066-48e4-a977-3ca304528e44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e335978146e4224b1306fa52a17ff4e'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:00.083897 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 888/1766] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:39:00 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:39:00.083897 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93026849-7b42-45a2-9e51-b6167099c303 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:00.094719 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93026849-7b42-45a2-9e51-b6167099c303 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:00.102579 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa362c6-950f-4deb-9bb5-23db5701f8cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:00.107760 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa362c6-950f-4deb-9bb5-23db5701f8cf admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:00.107760 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa362c6-950f-4deb-9bb5-23db5701f8cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:00.107760 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa362c6-950f-4deb-9bb5-23db5701f8cf admin admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:00.107760 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa362c6-950f-4deb-9bb5-23db5701f8cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:39:00.107760 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa362c6-950f-4deb-9bb5-23db5701f8cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:00.107760 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa362c6-950f-4deb-9bb5-23db5701f8cf admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:00.157629 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 889/1767] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 15 07:39:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:39:00.157814 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-587df126-4066-48e4-a977-3ca304528e44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e335978146e4224b1306fa52a17ff4e'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:00.157910 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa362c6-950f-4deb-9bb5-23db5701f8cf admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:00.158855 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-587df126-4066-48e4-a977-3ca304528e44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e335978146e4224b1306fa52a17ff4e'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:00.159817 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-dfa362c6-950f-4deb-9bb5-23db5701f8cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:00.170479 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-587df126-4066-48e4-a977-3ca304528e44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e335978146e4224b1306fa52a17ff4e'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:00.179234 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa362c6-950f-4deb-9bb5-23db5701f8cf admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:00.185925 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877c80cb-28cb-49cb-b18f-394eb8302247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:00.186734 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877c80cb-28cb-49cb-b18f-394eb8302247 admin admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:00.187087 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877c80cb-28cb-49cb-b18f-394eb8302247 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:00.187440 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877c80cb-28cb-49cb-b18f-394eb8302247 admin admin] PATH_INFO: `/v3/projects/453ffafb74604ac3bcc22a765b47681d/users/9ecdaf3f71fd46d192daad31832b9c03/roles/6bbb643e0d8d4bc0a9b27c26edf59216` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:00.216300 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877c80cb-28cb-49cb-b18f-394eb8302247 admin admin] RBAC: Authorizing `identity:create_grant(project_id=453ffafb74604ac3bcc22a765b47681d, user_id=9ecdaf3f71fd46d192daad31832b9c03, role_id=6bbb643e0d8d4bc0a9b27c26edf59216)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:00.219041 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877c80cb-28cb-49cb-b18f-394eb8302247 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:00.232442 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:00.243330 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 879/1768] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:38:59 2022] DELETE /identity/v3/users/5e335978146e4224b1306fa52a17ff4e => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:39:00.245213 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587df126-4066-48e4-a977-3ca304528e44 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:00.246897 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-587df126-4066-48e4-a977-3ca304528e44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:00.340714 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 890/1769] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:39:00 2022] PUT /identity/v3/projects/453ffafb74604ac3bcc22a765b47681d/users/9ecdaf3f71fd46d192daad31832b9c03/roles/6bbb643e0d8d4bc0a9b27c26edf59216 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:39:00.342239 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84d580a-57ee-4ef5-917f-df0f1c4c1428 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:00.346352 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84d580a-57ee-4ef5-917f-df0f1c4c1428 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:00.346352 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84d580a-57ee-4ef5-917f-df0f1c4c1428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:00.419674 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-e84d580a-57ee-4ef5-917f-df0f1c4c1428 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:39:00.421945 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-e84d580a-57ee-4ef5-917f-df0f1c4c1428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:00.463342 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587df126-4066-48e4-a977-3ca304528e44 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:00.472312 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:00.562079 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 891/1770] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:39:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:39:00.562981 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84d580a-57ee-4ef5-917f-df0f1c4c1428 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:00.588404 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84d580a-57ee-4ef5-917f-df0f1c4c1428 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:00.599315 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ed3e94-73f3-42f0-867e-3496855abd47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:00.611116 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ed3e94-73f3-42f0-867e-3496855abd47 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:00.611543 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ed3e94-73f3-42f0-867e-3496855abd47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:00.611643 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ed3e94-73f3-42f0-867e-3496855abd47 admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:00.612316 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ed3e94-73f3-42f0-867e-3496855abd47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:00.614011 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ed3e94-73f3-42f0-867e-3496855abd47 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:00.625938 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4547334a-2f97-4d1d-a196-4a0a4f29b6c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:00.626765 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4547334a-2f97-4d1d-a196-4a0a4f29b6c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:00.626892 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4547334a-2f97-4d1d-a196-4a0a4f29b6c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:00.627107 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4547334a-2f97-4d1d-a196-4a0a4f29b6c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:00.627934 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4547334a-2f97-4d1d-a196-4a0a4f29b6c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:00.635520 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 892/1771] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:39:00 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:39:00.636500 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ed3e94-73f3-42f0-867e-3496855abd47 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:00.640272 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4547334a-2f97-4d1d-a196-4a0a4f29b6c0 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:00.647490 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-c3ed3e94-73f3-42f0-867e-3496855abd47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:00.834744 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ed3e94-73f3-42f0-867e-3496855abd47 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:00.844364 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae200c6-e210-489a-92a5-d26fe500b54e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:00.844364 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae200c6-e210-489a-92a5-d26fe500b54e service neutron] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:00.844364 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae200c6-e210-489a-92a5-d26fe500b54e service neutron] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:00.844364 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae200c6-e210-489a-92a5-d26fe500b54e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:00.868052 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 880/1772] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:39:00 2022] POST /identity/v3/projects => generated 371 bytes in 624 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:39:00.869046 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4547334a-2f97-4d1d-a196-4a0a4f29b6c0 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:00.882912 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae200c6-e210-489a-92a5-d26fe500b54e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:00.887558 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae200c6-e210-489a-92a5-d26fe500b54e service neutron] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:00.910148 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 893/1773] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:39:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:39:00.910148 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae200c6-e210-489a-92a5-d26fe500b54e service neutron] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:00.924436 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae200c6-e210-489a-92a5-d26fe500b54e service neutron] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:00.942979 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:01.065343 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4547334a-2f97-4d1d-a196-4a0a4f29b6c0 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:01.067165 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe68051-0ed5-4f0f-9b3f-7f85004add1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:01.067904 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe68051-0ed5-4f0f-9b3f-7f85004add1b admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:01.068230 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe68051-0ed5-4f0f-9b3f-7f85004add1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:01.068983 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe68051-0ed5-4f0f-9b3f-7f85004add1b admin admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:01.070038 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe68051-0ed5-4f0f-9b3f-7f85004add1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:01.072248 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe68051-0ed5-4f0f-9b3f-7f85004add1b admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:01.084536 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7dc08f-c47c-4ef5-ba7d-ad96fc1570cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:01.087198 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7dc08f-c47c-4ef5-ba7d-ad96fc1570cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:01.088916 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7dc08f-c47c-4ef5-ba7d-ad96fc1570cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:01.089687 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7dc08f-c47c-4ef5-ba7d-ad96fc1570cd admin admin] PATH_INFO: `/v3/projects/453ffafb74604ac3bcc22a765b47681d/users/9ecdaf3f71fd46d192daad31832b9c03/roles/6bbb643e0d8d4bc0a9b27c26edf59216` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:01.096985 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7dc08f-c47c-4ef5-ba7d-ad96fc1570cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=453ffafb74604ac3bcc22a765b47681d, user_id=9ecdaf3f71fd46d192daad31832b9c03, role_id=6bbb643e0d8d4bc0a9b27c26edf59216)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:01.108049 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7dc08f-c47c-4ef5-ba7d-ad96fc1570cd admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:01.144451 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 881/1774] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:39:00 2022] PUT /identity/v3/projects/453ffafb74604ac3bcc22a765b47681d/users/9ecdaf3f71fd46d192daad31832b9c03/roles/6bbb643e0d8d4bc0a9b27c26edf59216 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:39:01.216880 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 894/1775] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:39:00 2022] POST /identity/v3/users => generated 286 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:39:01.221268 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7dc08f-c47c-4ef5-ba7d-ad96fc1570cd admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:01.324943 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7dc08f-c47c-4ef5-ba7d-ad96fc1570cd admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:01.332305 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce608f6-5a19-4277-a356-6b4523ff2813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:01.332305 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce608f6-5a19-4277-a356-6b4523ff2813 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:01.332913 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce608f6-5a19-4277-a356-6b4523ff2813 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:01.332913 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce608f6-5a19-4277-a356-6b4523ff2813 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:01.333339 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce608f6-5a19-4277-a356-6b4523ff2813 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:01.334949 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce608f6-5a19-4277-a356-6b4523ff2813 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:01.433993 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 882/1776] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:39:01 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 07:39:01.434207 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe68051-0ed5-4f0f-9b3f-7f85004add1b admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:01.434207 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe68051-0ed5-4f0f-9b3f-7f85004add1b admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:01.434207 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe46ec4-560b-4f24-866e-f8db7a1b12b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:01.434207 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe46ec4-560b-4f24-866e-f8db7a1b12b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:01.434707 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe46ec4-560b-4f24-866e-f8db7a1b12b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:01.434707 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe46ec4-560b-4f24-866e-f8db7a1b12b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:01.434707 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe46ec4-560b-4f24-866e-f8db7a1b12b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:01.434707 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe46ec4-560b-4f24-866e-f8db7a1b12b9 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:01.434707 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 895/1777] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:39:01 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:39:01.435095 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce608f6-5a19-4277-a356-6b4523ff2813 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:01.435095 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce608f6-5a19-4277-a356-6b4523ff2813 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:01.435095 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbef2c4-6a3b-4ad9-a05c-2c2ae4c8cd3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:01.435095 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbef2c4-6a3b-4ad9-a05c-2c2ae4c8cd3b admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:01.435591 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbef2c4-6a3b-4ad9-a05c-2c2ae4c8cd3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:01.435591 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbef2c4-6a3b-4ad9-a05c-2c2ae4c8cd3b admin admin] PATH_INFO: `/v3/projects/d30e27d7df724422adef4323d486bc29/users/af9195a31df14d6b883e75744cfab9d5/roles/053e12af84e943e699460a9d76c2072b` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:01.441413 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbef2c4-6a3b-4ad9-a05c-2c2ae4c8cd3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d30e27d7df724422adef4323d486bc29, user_id=af9195a31df14d6b883e75744cfab9d5, role_id=053e12af84e943e699460a9d76c2072b)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:01.456797 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbef2c4-6a3b-4ad9-a05c-2c2ae4c8cd3b admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:01.921922 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 883/1778] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:39:01 2022] PUT /identity/v3/projects/d30e27d7df724422adef4323d486bc29/users/af9195a31df14d6b883e75744cfab9d5/roles/053e12af84e943e699460a9d76c2072b => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 07:39:01.923628 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01339f5d-c6aa-4307-86c3-8589ff4587af None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:01.923930 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01339f5d-c6aa-4307-86c3-8589ff4587af None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:01.924258 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01339f5d-c6aa-4307-86c3-8589ff4587af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:02.050548 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-01339f5d-c6aa-4307-86c3-8589ff4587af None None] MFA Rules not processed for user `af9195a31df14d6b883e75744cfab9d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:39:02.050548 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-01339f5d-c6aa-4307-86c3-8589ff4587af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:02.205789 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 896/1779] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:39:01 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:39:02.206369 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbef2c4-6a3b-4ad9-a05c-2c2ae4c8cd3b admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:02.206456 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:02.400577 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbef2c4-6a3b-4ad9-a05c-2c2ae4c8cd3b admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:02.411828 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f426bf-8185-4c78-9f7b-e416674f75a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:02.412520 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f426bf-8185-4c78-9f7b-e416674f75a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:02.412800 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f426bf-8185-4c78-9f7b-e416674f75a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:02.413074 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f426bf-8185-4c78-9f7b-e416674f75a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:02.415514 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f426bf-8185-4c78-9f7b-e416674f75a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:02.419801 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f426bf-8185-4c78-9f7b-e416674f75a4 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:02.578218 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01339f5d-c6aa-4307-86c3-8589ff4587af None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:02.579602 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-01339f5d-c6aa-4307-86c3-8589ff4587af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:02.591808 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 884/1780] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:39:02 2022] POST /identity/v3/projects => generated 373 bytes in 388 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:39:02.597199 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f426bf-8185-4c78-9f7b-e416674f75a4 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:02.610669 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01339f5d-c6aa-4307-86c3-8589ff4587af None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:02.689945 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:02.744033 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f426bf-8185-4c78-9f7b-e416674f75a4 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:02.786475 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd30244-3e68-43f4-b2e2-92430f31518b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:02.786475 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd30244-3e68-43f4-b2e2-92430f31518b admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:02.786475 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd30244-3e68-43f4-b2e2-92430f31518b admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:02.786475 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd30244-3e68-43f4-b2e2-92430f31518b admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:02.786952 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd30244-3e68-43f4-b2e2-92430f31518b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:02.789856 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd30244-3e68-43f4-b2e2-92430f31518b admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:02.829952 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975789d0-0255-4fed-8f73-3ef7a1783400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:02.831610 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975789d0-0255-4fed-8f73-3ef7a1783400 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:02.832454 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975789d0-0255-4fed-8f73-3ef7a1783400 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:02.833084 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975789d0-0255-4fed-8f73-3ef7a1783400 admin admin] PATH_INFO: `/v3/projects/33cf9c15ba744a1fada8d251f1f4f02a` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:02.834897 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975789d0-0255-4fed-8f73-3ef7a1783400 admin admin] RBAC: Authorizing `identity:delete_project(project_id=33cf9c15ba744a1fada8d251f1f4f02a)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:02.838807 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975789d0-0255-4fed-8f73-3ef7a1783400 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:02.924293 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 885/1781] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:39:02 2022] POST /identity/v3/users => generated 293 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:39:02.928460 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd30244-3e68-43f4-b2e2-92430f31518b admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:02.958858 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd30244-3e68-43f4-b2e2-92430f31518b admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:02.958858 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36679d35-5607-481f-bdce-209ff97fc7b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:02.959200 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36679d35-5607-481f-bdce-209ff97fc7b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:02.959703 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36679d35-5607-481f-bdce-209ff97fc7b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:02.960804 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36679d35-5607-481f-bdce-209ff97fc7b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:02.962405 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36679d35-5607-481f-bdce-209ff97fc7b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:02.964089 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36679d35-5607-481f-bdce-209ff97fc7b9 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:02.996529 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 886/1782] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:39:02 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 07:39:02.997914 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36679d35-5607-481f-bdce-209ff97fc7b9 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:03.028673 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36679d35-5607-481f-bdce-209ff97fc7b9 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:03.051390 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-975789d0-0255-4fed-8f73-3ef7a1783400 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33cf9c15ba744a1fada8d251f1f4f02a', 'request_id': 'req-975789d0-0255-4fed-8f73-3ef7a1783400'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:03.071838 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-975789d0-0255-4fed-8f73-3ef7a1783400 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33cf9c15ba744a1fada8d251f1f4f02a', 'request_id': 'req-975789d0-0255-4fed-8f73-3ef7a1783400'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:03.176227 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6201241-8efb-4b03-b178-d3e681b4f78a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:03.177287 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6201241-8efb-4b03-b178-d3e681b4f78a admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:03.177916 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6201241-8efb-4b03-b178-d3e681b4f78a admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:03.178709 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6201241-8efb-4b03-b178-d3e681b4f78a admin admin] PATH_INFO: `/v3/projects/a1063e44fa68486690d81ed819fa54d6/users/630f750772db4997abf961e4acae3750/roles/6bbb643e0d8d4bc0a9b27c26edf59216` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:03.184811 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-975789d0-0255-4fed-8f73-3ef7a1783400 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33cf9c15ba744a1fada8d251f1f4f02a', 'request_id': 'req-975789d0-0255-4fed-8f73-3ef7a1783400'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:03.201707 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6201241-8efb-4b03-b178-d3e681b4f78a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1063e44fa68486690d81ed819fa54d6, user_id=630f750772db4997abf961e4acae3750, role_id=6bbb643e0d8d4bc0a9b27c26edf59216)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:03.203069 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-975789d0-0255-4fed-8f73-3ef7a1783400 admin admin] The token cache is being invalidate because project 33cf9c15ba744a1fada8d251f1f4f02a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:39:03.203909 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-975789d0-0255-4fed-8f73-3ef7a1783400 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:03.204880 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-975789d0-0255-4fed-8f73-3ef7a1783400 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:03.205276 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6201241-8efb-4b03-b178-d3e681b4f78a admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:03.207388 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 897/1783] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:39:02 2022] DELETE /identity/v3/projects/33cf9c15ba744a1fada8d251f1f4f02a => generated 0 bytes in 630 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 07:39:03.214080 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975789d0-0255-4fed-8f73-3ef7a1783400 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:03.216087 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-975789d0-0255-4fed-8f73-3ef7a1783400 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:03.232415 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975789d0-0255-4fed-8f73-3ef7a1783400 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:03.238179 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf9dcb7-7cf8-4814-807d-0688d0be3362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:03.238846 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf9dcb7-7cf8-4814-807d-0688d0be3362 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:03.239134 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf9dcb7-7cf8-4814-807d-0688d0be3362 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:03.239420 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf9dcb7-7cf8-4814-807d-0688d0be3362 admin admin] PATH_INFO: `/v3/users/c86981226dc940e589efaa310a53e6c5` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:03.240594 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf9dcb7-7cf8-4814-807d-0688d0be3362 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c86981226dc940e589efaa310a53e6c5)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:03.243378 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf9dcb7-7cf8-4814-807d-0688d0be3362 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:03.287870 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 887/1784] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:39:02 2022] PUT /identity/v3/projects/a1063e44fa68486690d81ed819fa54d6/users/630f750772db4997abf961e4acae3750/roles/6bbb643e0d8d4bc0a9b27c26edf59216 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:39:03.292461 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6201241-8efb-4b03-b178-d3e681b4f78a admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:03.295252 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-d6201241-8efb-4b03-b178-d3e681b4f78a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:03.427080 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6201241-8efb-4b03-b178-d3e681b4f78a admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:03.449131 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4c4e87-ad7a-4f53-a405-1244cae2015e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:03.449859 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4c4e87-ad7a-4f53-a405-1244cae2015e admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:03.450138 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4c4e87-ad7a-4f53-a405-1244cae2015e admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:03.450422 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4c4e87-ad7a-4f53-a405-1244cae2015e admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:03.451091 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4c4e87-ad7a-4f53-a405-1244cae2015e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:03.453658 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4c4e87-ad7a-4f53-a405-1244cae2015e admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:03.471114 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 888/1785] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:39:03 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 07:39:03.475664 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4c4e87-ad7a-4f53-a405-1244cae2015e admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:03.485120 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4c4e87-ad7a-4f53-a405-1244cae2015e admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:03.491260 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e845253-68fd-483d-b44c-1ee20029d8ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:03.492941 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e845253-68fd-483d-b44c-1ee20029d8ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:03.493159 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e845253-68fd-483d-b44c-1ee20029d8ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:03.493483 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e845253-68fd-483d-b44c-1ee20029d8ef admin admin] PATH_INFO: `/v3/projects/a1063e44fa68486690d81ed819fa54d6/users/630f750772db4997abf961e4acae3750/roles/6bbb643e0d8d4bc0a9b27c26edf59216` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:03.495435 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e845253-68fd-483d-b44c-1ee20029d8ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1063e44fa68486690d81ed819fa54d6, user_id=630f750772db4997abf961e4acae3750, role_id=6bbb643e0d8d4bc0a9b27c26edf59216)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:03.498300 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e845253-68fd-483d-b44c-1ee20029d8ef admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:03.531060 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 889/1786] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:39:03 2022] PUT /identity/v3/projects/a1063e44fa68486690d81ed819fa54d6/users/630f750772db4997abf961e4acae3750/roles/6bbb643e0d8d4bc0a9b27c26edf59216 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:39:03.542959 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e432f23-e7a1-4bde-a86c-af9cc4531f99 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:03.543247 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e432f23-e7a1-4bde-a86c-af9cc4531f99 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:03.543576 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e432f23-e7a1-4bde-a86c-af9cc4531f99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:03.589351 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:03.602120 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-3e432f23-e7a1-4bde-a86c-af9cc4531f99 None None] MFA Rules not processed for user `630f750772db4997abf961e4acae3750`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:39:03.602120 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-3e432f23-e7a1-4bde-a86c-af9cc4531f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:03.628396 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-ddf9dcb7-7cf8-4814-807d-0688d0be3362 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c86981226dc940e589efaa310a53e6c5'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:03.710576 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-ddf9dcb7-7cf8-4814-807d-0688d0be3362 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c86981226dc940e589efaa310a53e6c5'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:03.711783 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-ddf9dcb7-7cf8-4814-807d-0688d0be3362 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c86981226dc940e589efaa310a53e6c5'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:03.712859 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-ddf9dcb7-7cf8-4814-807d-0688d0be3362 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c86981226dc940e589efaa310a53e6c5'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:03.725121 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-ddf9dcb7-7cf8-4814-807d-0688d0be3362 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c86981226dc940e589efaa310a53e6c5'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:03.729296 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:03.754682 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 898/1787] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:39:03 2022] DELETE /identity/v3/users/c86981226dc940e589efaa310a53e6c5 => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:39:03.769402 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 890/1788] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:39:03 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 07:39:03.783668 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf9dcb7-7cf8-4814-807d-0688d0be3362 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:03.790313 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-ddf9dcb7-7cf8-4814-807d-0688d0be3362 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:03.926722 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf9dcb7-7cf8-4814-807d-0688d0be3362 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:03.939475 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a108dff-71e0-464f-a9d0-f0e01d1c8660 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecf6642b7b84d3c9756bb41ab3349d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:03.942110 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a108dff-71e0-464f-a9d0-f0e01d1c8660 service cinder] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:03.942110 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a108dff-71e0-464f-a9d0-f0e01d1c8660 service cinder] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:03.942110 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a108dff-71e0-464f-a9d0-f0e01d1c8660 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:04.092037 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a108dff-71e0-464f-a9d0-f0e01d1c8660 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:04.092795 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a108dff-71e0-464f-a9d0-f0e01d1c8660 service cinder] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:04.133988 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 899/1789] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:39:03 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:39:04.155088 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab250c2a-b44c-467b-99c5-d36dde1c5c53 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:04.155652 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab250c2a-b44c-467b-99c5-d36dde1c5c53 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:04.155652 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab250c2a-b44c-467b-99c5-d36dde1c5c53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:04.243989 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-ab250c2a-b44c-467b-99c5-d36dde1c5c53 None None] MFA Rules not processed for user `9ecdaf3f71fd46d192daad31832b9c03`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:39:04.245298 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-ab250c2a-b44c-467b-99c5-d36dde1c5c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:04.409129 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 891/1790] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:39:04 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:39:04.489762 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a108dff-71e0-464f-a9d0-f0e01d1c8660 service cinder] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:04.490367 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-7a108dff-71e0-464f-a9d0-f0e01d1c8660 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:04.645007 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a108dff-71e0-464f-a9d0-f0e01d1c8660 service cinder] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:04.680413 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc159bf2-e80b-4959-84b1-f05c0eb3e83f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:04.682239 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc159bf2-e80b-4959-84b1-f05c0eb3e83f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:04.684162 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc159bf2-e80b-4959-84b1-f05c0eb3e83f admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:04.685296 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc159bf2-e80b-4959-84b1-f05c0eb3e83f admin admin] PATH_INFO: `/v3/projects/3a481596310a44ef9667b1f960f75c02` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:04.688036 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc159bf2-e80b-4959-84b1-f05c0eb3e83f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a481596310a44ef9667b1f960f75c02)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:04.690637 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc159bf2-e80b-4959-84b1-f05c0eb3e83f admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:04.883328 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-cc159bf2-e80b-4959-84b1-f05c0eb3e83f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a481596310a44ef9667b1f960f75c02', 'request_id': 'req-cc159bf2-e80b-4959-84b1-f05c0eb3e83f'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:04.891551 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-cc159bf2-e80b-4959-84b1-f05c0eb3e83f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a481596310a44ef9667b1f960f75c02', 'request_id': 'req-cc159bf2-e80b-4959-84b1-f05c0eb3e83f'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:04.928398 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:05.013697 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-cc159bf2-e80b-4959-84b1-f05c0eb3e83f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a481596310a44ef9667b1f960f75c02', 'request_id': 'req-cc159bf2-e80b-4959-84b1-f05c0eb3e83f'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:05.028982 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-cc159bf2-e80b-4959-84b1-f05c0eb3e83f admin admin] The token cache is being invalidate because project 3a481596310a44ef9667b1f960f75c02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:39:05.029645 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-cc159bf2-e80b-4959-84b1-f05c0eb3e83f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:05.030524 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-cc159bf2-e80b-4959-84b1-f05c0eb3e83f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:05.032656 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 900/1791] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:39:04 2022] DELETE /identity/v3/projects/3a481596310a44ef9667b1f960f75c02 => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:39:05.038046 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1758d84c-dab2-4f7e-9a18-4375a4c5672b None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:05.038339 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1758d84c-dab2-4f7e-9a18-4375a4c5672b None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:05.039189 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1758d84c-dab2-4f7e-9a18-4375a4c5672b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:05.064557 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-1758d84c-dab2-4f7e-9a18-4375a4c5672b None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:39:05.066063 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-1758d84c-dab2-4f7e-9a18-4375a4c5672b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:05.090811 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:05.147111 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 892/1792] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:39:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:39:05.155396 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ed492-7d7b-430e-8270-76e84ca39dd7 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:05.159633 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ed492-7d7b-430e-8270-76e84ca39dd7 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:05.159633 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ed492-7d7b-430e-8270-76e84ca39dd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:05.180077 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-206ed492-7d7b-430e-8270-76e84ca39dd7 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:39:05.181675 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-206ed492-7d7b-430e-8270-76e84ca39dd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:05.190392 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 901/1793] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:39:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 07:39:05.194978 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1758d84c-dab2-4f7e-9a18-4375a4c5672b None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:05.205424 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1758d84c-dab2-4f7e-9a18-4375a4c5672b None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:05.211030 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd7bcd1-80da-4400-afcf-481440518103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:05.211745 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd7bcd1-80da-4400-afcf-481440518103 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:05.212206 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd7bcd1-80da-4400-afcf-481440518103 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:05.212484 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd7bcd1-80da-4400-afcf-481440518103 admin admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:05.213288 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd7bcd1-80da-4400-afcf-481440518103 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:39:05.213675 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd7bcd1-80da-4400-afcf-481440518103 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:05.215716 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd7bcd1-80da-4400-afcf-481440518103 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:05.229791 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 893/1794] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 15 07:39:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:39:05.233845 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206ed492-7d7b-430e-8270-76e84ca39dd7 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:05.243837 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206ed492-7d7b-430e-8270-76e84ca39dd7 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:05.246993 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd7bcd1-80da-4400-afcf-481440518103 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:05.249171 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-acd7bcd1-80da-4400-afcf-481440518103 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:05.250993 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41de9a5-e7b2-487f-8970-dff998d79e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:05.256777 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41de9a5-e7b2-487f-8970-dff998d79e5b admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:05.256777 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41de9a5-e7b2-487f-8970-dff998d79e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:05.256777 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41de9a5-e7b2-487f-8970-dff998d79e5b admin admin] PATH_INFO: `/v3/projects` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:05.256777 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41de9a5-e7b2-487f-8970-dff998d79e5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:05.256777 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41de9a5-e7b2-487f-8970-dff998d79e5b admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:05.266294 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd7bcd1-80da-4400-afcf-481440518103 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:05.276972 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d037664-745a-4eaa-9b0a-f614837c1e82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:05.277648 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d037664-745a-4eaa-9b0a-f614837c1e82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:05.278006 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d037664-745a-4eaa-9b0a-f614837c1e82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:05.278437 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d037664-745a-4eaa-9b0a-f614837c1e82 admin admin] PATH_INFO: `/v3/users/c2cb219f5ffa426b9f098859681e1fb3` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:05.279819 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d037664-745a-4eaa-9b0a-f614837c1e82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2cb219f5ffa426b9f098859681e1fb3)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:05.281853 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d037664-745a-4eaa-9b0a-f614837c1e82 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:05.553877 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 902/1795] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:39:05 2022] POST /identity/v3/projects => generated 405 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:39:05.558586 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41de9a5-e7b2-487f-8970-dff998d79e5b admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:05.645792 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41de9a5-e7b2-487f-8970-dff998d79e5b admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:05.651636 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e53c2e9-9d9f-4a0c-a2c4-b74279c88e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:05.652261 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e53c2e9-9d9f-4a0c-a2c4-b74279c88e81 admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:05.652589 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e53c2e9-9d9f-4a0c-a2c4-b74279c88e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:05.652804 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e53c2e9-9d9f-4a0c-a2c4-b74279c88e81 admin admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:05.653440 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e53c2e9-9d9f-4a0c-a2c4-b74279c88e81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:05.655455 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e53c2e9-9d9f-4a0c-a2c4-b74279c88e81 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:05.695945 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-0d037664-745a-4eaa-9b0a-f614837c1e82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2cb219f5ffa426b9f098859681e1fb3'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:05.696598 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-0d037664-745a-4eaa-9b0a-f614837c1e82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2cb219f5ffa426b9f098859681e1fb3'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:05.780007 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 903/1796] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:39:05 2022] POST /identity/v3/users => generated 303 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 07:39:05.781273 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62775353-6fd2-4884-924d-8ed063674980 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:05.781561 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62775353-6fd2-4884-924d-8ed063674980 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:05.781859 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62775353-6fd2-4884-924d-8ed063674980 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:05.783307 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 904/1797] 10.222.0.42 () {54 vars in 921 bytes} [Thu Sep 15 07:39:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:39:05.784017 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62775353-6fd2-4884-924d-8ed063674980 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:05.785851 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-62775353-6fd2-4884-924d-8ed063674980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:05.802101 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62775353-6fd2-4884-924d-8ed063674980 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:05.807054 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ab99f0-72d5-47f7-af8d-0612cb6ae7bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:05.807662 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ab99f0-72d5-47f7-af8d-0612cb6ae7bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:05.807946 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ab99f0-72d5-47f7-af8d-0612cb6ae7bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:05.808223 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ab99f0-72d5-47f7-af8d-0612cb6ae7bb admin admin] PATH_INFO: `/v3/users/af9195a31df14d6b883e75744cfab9d5` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:05.809352 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ab99f0-72d5-47f7-af8d-0612cb6ae7bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=af9195a31df14d6b883e75744cfab9d5)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:05.811405 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ab99f0-72d5-47f7-af8d-0612cb6ae7bb admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:05.819169 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-0d037664-745a-4eaa-9b0a-f614837c1e82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2cb219f5ffa426b9f098859681e1fb3'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:05.820010 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-0d037664-745a-4eaa-9b0a-f614837c1e82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2cb219f5ffa426b9f098859681e1fb3'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:05.832438 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-0d037664-745a-4eaa-9b0a-f614837c1e82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2cb219f5ffa426b9f098859681e1fb3'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:05.875054 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 894/1798] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:39:05 2022] DELETE /identity/v3/users/c2cb219f5ffa426b9f098859681e1fb3 => generated 0 bytes in 628 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:39:05.876365 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d037664-745a-4eaa-9b0a-f614837c1e82 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:05.877981 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-0d037664-745a-4eaa-9b0a-f614837c1e82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:05.965015 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d037664-745a-4eaa-9b0a-f614837c1e82 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:05.969767 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff85cfa0-4268-4d95-a8b9-98cfe9ebc94f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:05.970497 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff85cfa0-4268-4d95-a8b9-98cfe9ebc94f admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:05.970781 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff85cfa0-4268-4d95-a8b9-98cfe9ebc94f admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:05.971060 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff85cfa0-4268-4d95-a8b9-98cfe9ebc94f admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:05.971710 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff85cfa0-4268-4d95-a8b9-98cfe9ebc94f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:05.973303 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff85cfa0-4268-4d95-a8b9-98cfe9ebc94f admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:05.984214 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 895/1799] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:39:05 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:39:05.985454 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c22c684-1646-4524-9e2a-c374c1342937 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:05.985906 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c22c684-1646-4524-9e2a-c374c1342937 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:05.986543 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c22c684-1646-4524-9e2a-c374c1342937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:06.008662 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-8c22c684-1646-4524-9e2a-c374c1342937 None None] MFA Rules not processed for user `dd87846b9a134248b28e0521b0a880c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:39:06.009258 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-8c22c684-1646-4524-9e2a-c374c1342937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:06.035199 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-44ab99f0-72d5-47f7-af8d-0612cb6ae7bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af9195a31df14d6b883e75744cfab9d5'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:06.080312 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 896/1800] 10.222.0.42 () {58 vars in 1018 bytes} [Thu Sep 15 07:39:05 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:39:06.081933 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82433b11-40e6-4588-9a7a-ce6b0318c5e7 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:06.082341 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82433b11-40e6-4588-9a7a-ce6b0318c5e7 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:06.082624 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82433b11-40e6-4588-9a7a-ce6b0318c5e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:06.102947 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-82433b11-40e6-4588-9a7a-ce6b0318c5e7 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:39:06.104341 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-82433b11-40e6-4588-9a7a-ce6b0318c5e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:06.112116 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 897/1801] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:39:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:39:06.112785 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82433b11-40e6-4588-9a7a-ce6b0318c5e7 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:06.123286 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82433b11-40e6-4588-9a7a-ce6b0318c5e7 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:06.128430 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc198ce-7577-4491-954c-28ae8127073e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:06.129039 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc198ce-7577-4491-954c-28ae8127073e admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:06.129319 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc198ce-7577-4491-954c-28ae8127073e admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:06.129601 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc198ce-7577-4491-954c-28ae8127073e admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:06.130255 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc198ce-7577-4491-954c-28ae8127073e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:06.131851 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc198ce-7577-4491-954c-28ae8127073e admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:06.134420 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-44ab99f0-72d5-47f7-af8d-0612cb6ae7bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af9195a31df14d6b883e75744cfab9d5'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:06.137163 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-44ab99f0-72d5-47f7-af8d-0612cb6ae7bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af9195a31df14d6b883e75744cfab9d5'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:06.137163 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-44ab99f0-72d5-47f7-af8d-0612cb6ae7bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af9195a31df14d6b883e75744cfab9d5'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:06.145291 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-44ab99f0-72d5-47f7-af8d-0612cb6ae7bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af9195a31df14d6b883e75744cfab9d5'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:06.146146 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 898/1802] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:39:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:39:06.147628 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e1edec-3308-4352-91e3-bfeb5887b484 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:06.148441 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e1edec-3308-4352-91e3-bfeb5887b484 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:06.148796 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e1edec-3308-4352-91e3-bfeb5887b484 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:06.150440 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 899/1803] 10.222.0.42 () {54 vars in 961 bytes} [Thu Sep 15 07:39:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:39:06.151232 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e1edec-3308-4352-91e3-bfeb5887b484 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:06.157208 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-23e1edec-3308-4352-91e3-bfeb5887b484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:06.198724 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 905/1804] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:39:05 2022] DELETE /identity/v3/users/af9195a31df14d6b883e75744cfab9d5 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:39:06.202842 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ab99f0-72d5-47f7-af8d-0612cb6ae7bb admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:06.204299 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-44ab99f0-72d5-47f7-af8d-0612cb6ae7bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:06.230471 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e1edec-3308-4352-91e3-bfeb5887b484 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:06.275809 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ab99f0-72d5-47f7-af8d-0612cb6ae7bb admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:06.281902 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee3c07d-8255-433a-af7a-efb2cdb1a851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:06.282699 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee3c07d-8255-433a-af7a-efb2cdb1a851 admin admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:06.283031 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee3c07d-8255-433a-af7a-efb2cdb1a851 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:06.283363 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee3c07d-8255-433a-af7a-efb2cdb1a851 admin admin] PATH_INFO: `/v3/projects/673b70ad99494879b805f96da622b1f4/users/d33704412b3245fc9b31395f0611483f/roles/053e12af84e943e699460a9d76c2072b` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:06.286908 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11f4a9f-2205-455c-8427-65d7e89e4c24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:06.288081 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11f4a9f-2205-455c-8427-65d7e89e4c24 service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:06.288081 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11f4a9f-2205-455c-8427-65d7e89e4c24 service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:06.288081 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11f4a9f-2205-455c-8427-65d7e89e4c24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:06.290144 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-b11f4a9f-2205-455c-8427-65d7e89e4c24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:06.299878 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee3c07d-8255-433a-af7a-efb2cdb1a851 admin admin] RBAC: Authorizing `identity:create_grant(project_id=673b70ad99494879b805f96da622b1f4, user_id=d33704412b3245fc9b31395f0611483f, role_id=053e12af84e943e699460a9d76c2072b)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:06.302539 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11f4a9f-2205-455c-8427-65d7e89e4c24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:06.302813 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee3c07d-8255-433a-af7a-efb2cdb1a851 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:06.303985 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11f4a9f-2205-455c-8427-65d7e89e4c24 service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:06.314289 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 900/1805] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:39:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:39:06.315513 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9b7a6e-ba4b-43ba-8879-8a42e8df5a2c None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:06.315824 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9b7a6e-ba4b-43ba-8879-8a42e8df5a2c None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:06.316108 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9b7a6e-ba4b-43ba-8879-8a42e8df5a2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:06.328776 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:06.344590 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-1e9b7a6e-ba4b-43ba-8879-8a42e8df5a2c None None] MFA Rules not processed for user `dd87846b9a134248b28e0521b0a880c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:39:06.346082 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-1e9b7a6e-ba4b-43ba-8879-8a42e8df5a2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:06.393262 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 906/1806] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:39:06 2022] PUT /identity/v3/projects/673b70ad99494879b805f96da622b1f4/users/d33704412b3245fc9b31395f0611483f/roles/053e12af84e943e699460a9d76c2072b => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:39:06.395035 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b63aaf2-1878-4f40-926a-c1a3b2cf2da3 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:06.395472 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b63aaf2-1878-4f40-926a-c1a3b2cf2da3 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:06.395907 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b63aaf2-1878-4f40-926a-c1a3b2cf2da3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:06.428610 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-4b63aaf2-1878-4f40-926a-c1a3b2cf2da3 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:39:06.433406 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-4b63aaf2-1878-4f40-926a-c1a3b2cf2da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:06.475693 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 901/1807] 10.222.0.42 () {58 vars in 1058 bytes} [Thu Sep 15 07:39:06 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:39:06.485520 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76954f9d-5c1e-498b-92df-2daa7d0dce7f None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:06.488581 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76954f9d-5c1e-498b-92df-2daa7d0dce7f None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:06.488581 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76954f9d-5c1e-498b-92df-2daa7d0dce7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:06.556209 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 907/1808] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:39:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:39:06.556209 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b63aaf2-1878-4f40-926a-c1a3b2cf2da3 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:06.583719 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-76954f9d-5c1e-498b-92df-2daa7d0dce7f None None] MFA Rules not processed for user `d33704412b3245fc9b31395f0611483f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:39:06.587075 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-76954f9d-5c1e-498b-92df-2daa7d0dce7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:06.594770 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=79,keystone:DELETE=15,keystone:INSERT=6 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:06.696097 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b63aaf2-1878-4f40-926a-c1a3b2cf2da3 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:06.702328 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194c9721-f98b-4ac5-b6aa-60baef08b7c1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd87846b9a134248b28e0521b0a880c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:06.702893 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194c9721-f98b-4ac5-b6aa-60baef08b7c1 service placement] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:06.703171 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194c9721-f98b-4ac5-b6aa-60baef08b7c1 service placement] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:06.704095 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194c9721-f98b-4ac5-b6aa-60baef08b7c1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:06.706456 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-194c9721-f98b-4ac5-b6aa-60baef08b7c1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:06.721059 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194c9721-f98b-4ac5-b6aa-60baef08b7c1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:06.721545 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 902/1809] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:39:06 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:39:06.723001 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76954f9d-5c1e-498b-92df-2daa7d0dce7f None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:06.723736 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194c9721-f98b-4ac5-b6aa-60baef08b7c1 service placement] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:06.724971 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-76954f9d-5c1e-498b-92df-2daa7d0dce7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:06.737198 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 908/1810] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:39:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:39:06.748275 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194c9721-f98b-4ac5-b6aa-60baef08b7c1 service placement] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:06.764232 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194c9721-f98b-4ac5-b6aa-60baef08b7c1 service placement] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:06.769975 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dcf216-9b47-42e4-8622-f2bb3f521cb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:06.770794 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dcf216-9b47-42e4-8622-f2bb3f521cb1 admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:06.771181 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dcf216-9b47-42e4-8622-f2bb3f521cb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:06.771555 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dcf216-9b47-42e4-8622-f2bb3f521cb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:06.772790 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dcf216-9b47-42e4-8622-f2bb3f521cb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:06.776232 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dcf216-9b47-42e4-8622-f2bb3f521cb1 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:06.840813 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76954f9d-5c1e-498b-92df-2daa7d0dce7f None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:06.845963 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3866d5-c978-4885-86a7-8cdfcdfb2d98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:06.846418 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3866d5-c978-4885-86a7-8cdfcdfb2d98 service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:06.846865 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3866d5-c978-4885-86a7-8cdfcdfb2d98 service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:06.847144 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3866d5-c978-4885-86a7-8cdfcdfb2d98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:06.866419 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3866d5-c978-4885-86a7-8cdfcdfb2d98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:06.868362 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3866d5-c978-4885-86a7-8cdfcdfb2d98 service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:06.879889 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 903/1811] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:39:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:39:06.880765 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3866d5-c978-4885-86a7-8cdfcdfb2d98 service neutron] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:06.882094 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-ef3866d5-c978-4885-86a7-8cdfcdfb2d98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:06.896404 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3866d5-c978-4885-86a7-8cdfcdfb2d98 service neutron] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:06.902416 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7026fa-ee60-49ac-a8d9-3d273f9d716a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:06.903120 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7026fa-ee60-49ac-a8d9-3d273f9d716a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:06.903407 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7026fa-ee60-49ac-a8d9-3d273f9d716a admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:06.903731 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7026fa-ee60-49ac-a8d9-3d273f9d716a admin admin] PATH_INFO: `/v3/users/056886a1210144f1ac838ac94b6eac6e` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:06.904936 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7026fa-ee60-49ac-a8d9-3d273f9d716a admin admin] RBAC: Authorizing `identity:delete_user(user_id=056886a1210144f1ac838ac94b6eac6e)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:06.907276 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7026fa-ee60-49ac-a8d9-3d273f9d716a admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:06.928940 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 909/1812] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:39:06 2022] POST /identity/v3/projects => generated 407 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:39:06.931374 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dcf216-9b47-42e4-8622-f2bb3f521cb1 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:07.057999 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dcf216-9b47-42e4-8622-f2bb3f521cb1 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:07.062755 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d343272b-9ffc-4909-8fd0-c5708d0d6179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:07.070965 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d343272b-9ffc-4909-8fd0-c5708d0d6179 admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:07.070965 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d343272b-9ffc-4909-8fd0-c5708d0d6179 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:07.070965 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d343272b-9ffc-4909-8fd0-c5708d0d6179 admin admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:07.070965 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d343272b-9ffc-4909-8fd0-c5708d0d6179 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:07.070965 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d343272b-9ffc-4909-8fd0-c5708d0d6179 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:07.155244 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-9a7026fa-ee60-49ac-a8d9-3d273f9d716a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '056886a1210144f1ac838ac94b6eac6e'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:07.156161 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-9a7026fa-ee60-49ac-a8d9-3d273f9d716a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '056886a1210144f1ac838ac94b6eac6e'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:07.161623 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 910/1813] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:39:06 2022] POST /identity/v3/users => generated 310 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:39:07.163422 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d343272b-9ffc-4909-8fd0-c5708d0d6179 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:07.167616 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-d343272b-9ffc-4909-8fd0-c5708d0d6179 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:07.180772 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d343272b-9ffc-4909-8fd0-c5708d0d6179 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:07.185496 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5264da97-842c-417f-8c35-415137022603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:07.186174 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5264da97-842c-417f-8c35-415137022603 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:07.188222 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5264da97-842c-417f-8c35-415137022603 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:07.188222 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5264da97-842c-417f-8c35-415137022603 admin admin] PATH_INFO: `/v3/projects/dee2984dda404692a5058904534b449c` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:07.188222 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5264da97-842c-417f-8c35-415137022603 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dee2984dda404692a5058904534b449c)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:07.189859 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5264da97-842c-417f-8c35-415137022603 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:07.301631 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-9a7026fa-ee60-49ac-a8d9-3d273f9d716a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '056886a1210144f1ac838ac94b6eac6e'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:07.302260 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-9a7026fa-ee60-49ac-a8d9-3d273f9d716a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '056886a1210144f1ac838ac94b6eac6e'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:07.320762 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-9a7026fa-ee60-49ac-a8d9-3d273f9d716a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '056886a1210144f1ac838ac94b6eac6e'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:07.349300 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 904/1814] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:39:06 2022] DELETE /identity/v3/users/056886a1210144f1ac838ac94b6eac6e => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:39:07.353258 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7026fa-ee60-49ac-a8d9-3d273f9d716a admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:07.353435 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-5264da97-842c-417f-8c35-415137022603 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dee2984dda404692a5058904534b449c', 'request_id': 'req-5264da97-842c-417f-8c35-415137022603'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:07.356469 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-9a7026fa-ee60-49ac-a8d9-3d273f9d716a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:07.373618 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-5264da97-842c-417f-8c35-415137022603 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dee2984dda404692a5058904534b449c', 'request_id': 'req-5264da97-842c-417f-8c35-415137022603'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:07.471141 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7026fa-ee60-49ac-a8d9-3d273f9d716a admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:07.478429 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b475d9-c5fb-4422-b9d4-f955d201e369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:07.479138 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b475d9-c5fb-4422-b9d4-f955d201e369 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:07.479487 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b475d9-c5fb-4422-b9d4-f955d201e369 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:07.479826 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b475d9-c5fb-4422-b9d4-f955d201e369 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:07.480607 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b475d9-c5fb-4422-b9d4-f955d201e369 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:07.482316 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b475d9-c5fb-4422-b9d4-f955d201e369 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:07.502262 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 905/1815] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:39:07 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:39:07.505473 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b475d9-c5fb-4422-b9d4-f955d201e369 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:07.519788 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b475d9-c5fb-4422-b9d4-f955d201e369 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:07.525499 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d360a1-46ef-496b-86a2-3ee9c849539b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:07.526340 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d360a1-46ef-496b-86a2-3ee9c849539b admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:07.532529 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d360a1-46ef-496b-86a2-3ee9c849539b admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:07.532529 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d360a1-46ef-496b-86a2-3ee9c849539b admin admin] PATH_INFO: `/v3/projects/a2f8eb7aec3c436fbfb1a0eb8d3df0b6/users/d359f1820758445da420a679d3ad7544/roles/6bbb643e0d8d4bc0a9b27c26edf59216` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:07.532742 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-5264da97-842c-417f-8c35-415137022603 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dee2984dda404692a5058904534b449c', 'request_id': 'req-5264da97-842c-417f-8c35-415137022603'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:07.543960 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d360a1-46ef-496b-86a2-3ee9c849539b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2f8eb7aec3c436fbfb1a0eb8d3df0b6, user_id=d359f1820758445da420a679d3ad7544, role_id=6bbb643e0d8d4bc0a9b27c26edf59216)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:07.550109 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d360a1-46ef-496b-86a2-3ee9c849539b admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:07.558485 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-5264da97-842c-417f-8c35-415137022603 admin admin] The token cache is being invalidate because project dee2984dda404692a5058904534b449c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:39:07.558485 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-5264da97-842c-417f-8c35-415137022603 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:07.558485 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-5264da97-842c-417f-8c35-415137022603 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:07.563028 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 911/1816] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:39:07 2022] DELETE /identity/v3/projects/dee2984dda404692a5058904534b449c => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:39:07.564922 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77706a00-4881-4741-8776-e4d1a9a48204 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:07.564922 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77706a00-4881-4741-8776-e4d1a9a48204 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:07.565143 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77706a00-4881-4741-8776-e4d1a9a48204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:07.612724 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-77706a00-4881-4741-8776-e4d1a9a48204 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:39:07.613607 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-77706a00-4881-4741-8776-e4d1a9a48204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:07.635516 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 912/1817] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:39:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:39:07.642579 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728a6cd8-ef34-4a5e-af61-7e3248513661 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:07.642996 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728a6cd8-ef34-4a5e-af61-7e3248513661 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:07.643423 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728a6cd8-ef34-4a5e-af61-7e3248513661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:07.670975 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-728a6cd8-ef34-4a5e-af61-7e3248513661 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:39:07.672241 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-728a6cd8-ef34-4a5e-af61-7e3248513661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:07.695383 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 906/1818] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:39:07 2022] PUT /identity/v3/projects/a2f8eb7aec3c436fbfb1a0eb8d3df0b6/users/d359f1820758445da420a679d3ad7544/roles/6bbb643e0d8d4bc0a9b27c26edf59216 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 07:39:07.696293 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 913/1819] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:39:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 07:39:07.702057 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728a6cd8-ef34-4a5e-af61-7e3248513661 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:07.702938 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d360a1-46ef-496b-86a2-3ee9c849539b admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:07.705468 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-02d360a1-46ef-496b-86a2-3ee9c849539b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:07.719147 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728a6cd8-ef34-4a5e-af61-7e3248513661 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:07.806854 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d360a1-46ef-496b-86a2-3ee9c849539b admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:07.814316 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2121125-c061-4038-960a-3e70beb75929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:07.821756 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2121125-c061-4038-960a-3e70beb75929 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:07.822239 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2121125-c061-4038-960a-3e70beb75929 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:07.827331 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a8db57-6381-4f39-8b21-0f6d13e9a916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:07.827468 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2121125-c061-4038-960a-3e70beb75929 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:07.827560 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a8db57-6381-4f39-8b21-0f6d13e9a916 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:07.827560 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a8db57-6381-4f39-8b21-0f6d13e9a916 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:07.827717 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2121125-c061-4038-960a-3e70beb75929 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:07.827798 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a8db57-6381-4f39-8b21-0f6d13e9a916 admin admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:07.827798 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a8db57-6381-4f39-8b21-0f6d13e9a916 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:39:07.827798 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a8db57-6381-4f39-8b21-0f6d13e9a916 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:07.828005 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2121125-c061-4038-960a-3e70beb75929 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:07.828089 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a8db57-6381-4f39-8b21-0f6d13e9a916 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:07.838873 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 907/1820] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:39:07 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:39:07.842886 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2121125-c061-4038-960a-3e70beb75929 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:07.856459 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 914/1821] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 15 07:39:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:39:07.858071 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2121125-c061-4038-960a-3e70beb75929 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:07.860289 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a8db57-6381-4f39-8b21-0f6d13e9a916 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:07.873148 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bc0cf7-aa57-4083-99e1-4571676af73a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:07.873864 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bc0cf7-aa57-4083-99e1-4571676af73a admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:07.874479 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bc0cf7-aa57-4083-99e1-4571676af73a admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:07.874836 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bc0cf7-aa57-4083-99e1-4571676af73a admin admin] PATH_INFO: `/v3/projects/a2f8eb7aec3c436fbfb1a0eb8d3df0b6/users/d359f1820758445da420a679d3ad7544/roles/6bbb643e0d8d4bc0a9b27c26edf59216` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:07.877472 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bc0cf7-aa57-4083-99e1-4571676af73a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2f8eb7aec3c436fbfb1a0eb8d3df0b6, user_id=d359f1820758445da420a679d3ad7544, role_id=6bbb643e0d8d4bc0a9b27c26edf59216)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:07.878449 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a8db57-6381-4f39-8b21-0f6d13e9a916 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:07.887272 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:07.887272 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bc0cf7-aa57-4083-99e1-4571676af73a admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:07.887474 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8298b736-5264-4e54-b7e1-fbdcaae98e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:07.887474 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8298b736-5264-4e54-b7e1-fbdcaae98e76 admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:07.887474 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8298b736-5264-4e54-b7e1-fbdcaae98e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:07.887474 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8298b736-5264-4e54-b7e1-fbdcaae98e76 admin admin] PATH_INFO: `/v3/projects` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:07.887775 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8298b736-5264-4e54-b7e1-fbdcaae98e76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:07.888783 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8298b736-5264-4e54-b7e1-fbdcaae98e76 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:07.900348 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 908/1822] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:39:07 2022] PUT /identity/v3/projects/a2f8eb7aec3c436fbfb1a0eb8d3df0b6/users/d359f1820758445da420a679d3ad7544/roles/6bbb643e0d8d4bc0a9b27c26edf59216 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:39:07.901677 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bc0cf7-aa57-4083-99e1-4571676af73a admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:07.910767 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-38bc0cf7-aa57-4083-99e1-4571676af73a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:08.001012 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 915/1823] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:39:07 2022] POST /identity/v3/projects => generated 373 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:39:08.002541 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427bede5-a3dc-4132-8cef-1c79c98c3828 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:08.002828 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427bede5-a3dc-4132-8cef-1c79c98c3828 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:08.003323 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427bede5-a3dc-4132-8cef-1c79c98c3828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:08.011643 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bc0cf7-aa57-4083-99e1-4571676af73a admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:08.050626 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-427bede5-a3dc-4132-8cef-1c79c98c3828 None None] MFA Rules not processed for user `d359f1820758445da420a679d3ad7544`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:39:08.052186 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-427bede5-a3dc-4132-8cef-1c79c98c3828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:08.098604 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b146f410-3835-408b-8211-91fedd6eaee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:08.099388 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b146f410-3835-408b-8211-91fedd6eaee3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:08.099727 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b146f410-3835-408b-8211-91fedd6eaee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:08.100010 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b146f410-3835-408b-8211-91fedd6eaee3 admin admin] PATH_INFO: `/v3/projects/d30e27d7df724422adef4323d486bc29` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:08.101707 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b146f410-3835-408b-8211-91fedd6eaee3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d30e27d7df724422adef4323d486bc29)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:08.106784 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b146f410-3835-408b-8211-91fedd6eaee3 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:08.186787 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 916/1824] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:39:08 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:39:08.216348 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427bede5-a3dc-4132-8cef-1c79c98c3828 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:08.216348 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-427bede5-a3dc-4132-8cef-1c79c98c3828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:08.216348 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427bede5-a3dc-4132-8cef-1c79c98c3828 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:08.218752 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86d8102-1663-4efa-9101-4fc34eb62544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:08.218752 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86d8102-1663-4efa-9101-4fc34eb62544 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:08.218752 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86d8102-1663-4efa-9101-4fc34eb62544 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:08.218752 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86d8102-1663-4efa-9101-4fc34eb62544 admin admin] PATH_INFO: `/v3/projects/3254ca4f60ec4def824456f9887dde02` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:08.219152 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86d8102-1663-4efa-9101-4fc34eb62544 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3254ca4f60ec4def824456f9887dde02)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:08.219152 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86d8102-1663-4efa-9101-4fc34eb62544 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:08.308564 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-b146f410-3835-408b-8211-91fedd6eaee3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd30e27d7df724422adef4323d486bc29', 'request_id': 'req-b146f410-3835-408b-8211-91fedd6eaee3'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:08.318693 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-b146f410-3835-408b-8211-91fedd6eaee3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd30e27d7df724422adef4323d486bc29', 'request_id': 'req-b146f410-3835-408b-8211-91fedd6eaee3'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:08.385486 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-a86d8102-1663-4efa-9101-4fc34eb62544 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3254ca4f60ec4def824456f9887dde02', 'request_id': 'req-a86d8102-1663-4efa-9101-4fc34eb62544'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:08.392665 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-a86d8102-1663-4efa-9101-4fc34eb62544 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3254ca4f60ec4def824456f9887dde02', 'request_id': 'req-a86d8102-1663-4efa-9101-4fc34eb62544'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:08.401444 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-b146f410-3835-408b-8211-91fedd6eaee3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd30e27d7df724422adef4323d486bc29', 'request_id': 'req-b146f410-3835-408b-8211-91fedd6eaee3'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:08.415601 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-b146f410-3835-408b-8211-91fedd6eaee3 admin admin] The token cache is being invalidate because project d30e27d7df724422adef4323d486bc29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:39:08.415930 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-b146f410-3835-408b-8211-91fedd6eaee3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:08.416599 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-b146f410-3835-408b-8211-91fedd6eaee3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:08.418349 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 909/1825] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:39:07 2022] DELETE /identity/v3/projects/d30e27d7df724422adef4323d486bc29 => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:39:08.419198 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b146f410-3835-408b-8211-91fedd6eaee3 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:08.420712 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-b146f410-3835-408b-8211-91fedd6eaee3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:08.504766 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-a86d8102-1663-4efa-9101-4fc34eb62544 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3254ca4f60ec4def824456f9887dde02', 'request_id': 'req-a86d8102-1663-4efa-9101-4fc34eb62544'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:08.518482 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:08.519492 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b146f410-3835-408b-8211-91fedd6eaee3 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:08.531066 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-a86d8102-1663-4efa-9101-4fc34eb62544 admin admin] The token cache is being invalidate because project 3254ca4f60ec4def824456f9887dde02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:39:08.531350 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-a86d8102-1663-4efa-9101-4fc34eb62544 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:08.533053 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-a86d8102-1663-4efa-9101-4fc34eb62544 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:08.535054 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 917/1826] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:39:08 2022] DELETE /identity/v3/projects/3254ca4f60ec4def824456f9887dde02 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:39:08.537158 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86d8102-1663-4efa-9101-4fc34eb62544 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:08.540153 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-a86d8102-1663-4efa-9101-4fc34eb62544 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:08.598517 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:08.601009 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbd4977-af92-4064-a78f-cb9f2cfe7556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:08.602013 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbd4977-af92-4064-a78f-cb9f2cfe7556 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:08.602568 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbd4977-af92-4064-a78f-cb9f2cfe7556 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:08.603326 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbd4977-af92-4064-a78f-cb9f2cfe7556 admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:08.604022 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbd4977-af92-4064-a78f-cb9f2cfe7556 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:08.606021 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbd4977-af92-4064-a78f-cb9f2cfe7556 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:08.644605 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86d8102-1663-4efa-9101-4fc34eb62544 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:08.649875 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3c07ad-1a94-47cc-8ae5-39f8933a6372 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecf6642b7b84d3c9756bb41ab3349d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:08.650603 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3c07ad-1a94-47cc-8ae5-39f8933a6372 service cinder] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:08.650886 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3c07ad-1a94-47cc-8ae5-39f8933a6372 service cinder] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:08.651167 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3c07ad-1a94-47cc-8ae5-39f8933a6372 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:08.653187 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-5d3c07ad-1a94-47cc-8ae5-39f8933a6372 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:08.714978 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 910/1827] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:39:08 2022] POST /identity/v3/users => generated 287 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:39:08.714978 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbd4977-af92-4064-a78f-cb9f2cfe7556 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:08.716858 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-fcbd4977-af92-4064-a78f-cb9f2cfe7556 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:08.745631 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbd4977-af92-4064-a78f-cb9f2cfe7556 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:08.753104 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3c07ad-1a94-47cc-8ae5-39f8933a6372 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:08.753279 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbbce35-0608-48bb-8d9d-4d800a1d6712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:08.753760 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbbce35-0608-48bb-8d9d-4d800a1d6712 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:08.754366 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbbce35-0608-48bb-8d9d-4d800a1d6712 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:08.754806 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbbce35-0608-48bb-8d9d-4d800a1d6712 admin admin] PATH_INFO: `/v3/users/630f750772db4997abf961e4acae3750` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:08.755102 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3c07ad-1a94-47cc-8ae5-39f8933a6372 service cinder] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:08.755965 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbbce35-0608-48bb-8d9d-4d800a1d6712 admin admin] RBAC: Authorizing `identity:delete_user(user_id=630f750772db4997abf961e4acae3750)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:08.758742 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbbce35-0608-48bb-8d9d-4d800a1d6712 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:08.781947 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 918/1828] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:39:08 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:39:08.782783 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3c07ad-1a94-47cc-8ae5-39f8933a6372 service cinder] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:08.784470 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-5d3c07ad-1a94-47cc-8ae5-39f8933a6372 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:08.803991 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3c07ad-1a94-47cc-8ae5-39f8933a6372 service cinder] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:08.810693 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab582a9-04e1-4e64-8fd2-b8c2ded9e4a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:08.812268 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab582a9-04e1-4e64-8fd2-b8c2ded9e4a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:08.812382 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab582a9-04e1-4e64-8fd2-b8c2ded9e4a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:08.812970 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab582a9-04e1-4e64-8fd2-b8c2ded9e4a7 admin admin] PATH_INFO: `/v3/users/9ecdaf3f71fd46d192daad31832b9c03` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:08.814348 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab582a9-04e1-4e64-8fd2-b8c2ded9e4a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ecdaf3f71fd46d192daad31832b9c03)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:08.817454 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab582a9-04e1-4e64-8fd2-b8c2ded9e4a7 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:09.058710 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-9bbbce35-0608-48bb-8d9d-4d800a1d6712 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '630f750772db4997abf961e4acae3750'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:09.059378 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-9bbbce35-0608-48bb-8d9d-4d800a1d6712 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '630f750772db4997abf961e4acae3750'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:09.140115 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-9bbbce35-0608-48bb-8d9d-4d800a1d6712 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '630f750772db4997abf961e4acae3750'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:09.141569 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-9bbbce35-0608-48bb-8d9d-4d800a1d6712 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '630f750772db4997abf961e4acae3750'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:09.152138 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-9bbbce35-0608-48bb-8d9d-4d800a1d6712 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '630f750772db4997abf961e4acae3750'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:09.158577 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-3ab582a9-04e1-4e64-8fd2-b8c2ded9e4a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ecdaf3f71fd46d192daad31832b9c03'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:09.171762 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 911/1829] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:39:08 2022] DELETE /identity/v3/users/630f750772db4997abf961e4acae3750 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:39:09.173234 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbbce35-0608-48bb-8d9d-4d800a1d6712 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:09.174847 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-9bbbce35-0608-48bb-8d9d-4d800a1d6712 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:09.244418 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-3ab582a9-04e1-4e64-8fd2-b8c2ded9e4a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ecdaf3f71fd46d192daad31832b9c03'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:09.246315 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-3ab582a9-04e1-4e64-8fd2-b8c2ded9e4a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ecdaf3f71fd46d192daad31832b9c03'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:09.247312 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-3ab582a9-04e1-4e64-8fd2-b8c2ded9e4a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ecdaf3f71fd46d192daad31832b9c03'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:09.268800 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-3ab582a9-04e1-4e64-8fd2-b8c2ded9e4a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ecdaf3f71fd46d192daad31832b9c03'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:09.286176 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbbce35-0608-48bb-8d9d-4d800a1d6712 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:09.297042 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8296166c-f914-43dd-90bc-8c78c39d9698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:09.299204 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8296166c-f914-43dd-90bc-8c78c39d9698 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:09.299494 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8296166c-f914-43dd-90bc-8c78c39d9698 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:09.299774 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8296166c-f914-43dd-90bc-8c78c39d9698 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:09.300442 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8296166c-f914-43dd-90bc-8c78c39d9698 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:09.302113 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8296166c-f914-43dd-90bc-8c78c39d9698 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:09.316432 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 919/1830] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:39:08 2022] DELETE /identity/v3/users/9ecdaf3f71fd46d192daad31832b9c03 => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:39:09.316432 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab582a9-04e1-4e64-8fd2-b8c2ded9e4a7 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:09.317671 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-3ab582a9-04e1-4e64-8fd2-b8c2ded9e4a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:09.322885 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 912/1831] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:39:09 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:39:09.329863 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8296166c-f914-43dd-90bc-8c78c39d9698 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:09.334616 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-8296166c-f914-43dd-90bc-8c78c39d9698 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:09.373970 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8296166c-f914-43dd-90bc-8c78c39d9698 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:09.536021 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab582a9-04e1-4e64-8fd2-b8c2ded9e4a7 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:09.564539 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589f77c4-a7d7-4782-8747-8c8544acd30d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecf6642b7b84d3c9756bb41ab3349d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:09.568587 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589f77c4-a7d7-4782-8747-8c8544acd30d service cinder] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:09.568811 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589f77c4-a7d7-4782-8747-8c8544acd30d service cinder] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:09.569099 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589f77c4-a7d7-4782-8747-8c8544acd30d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:09.576033 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-589f77c4-a7d7-4782-8747-8c8544acd30d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:09.604179 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9469b08-8a43-4511-ab8b-223d8bff43a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:09.604977 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9469b08-8a43-4511-ab8b-223d8bff43a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:09.607174 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9469b08-8a43-4511-ab8b-223d8bff43a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:09.607426 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9469b08-8a43-4511-ab8b-223d8bff43a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:09.608109 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9469b08-8a43-4511-ab8b-223d8bff43a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:09.616475 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9469b08-8a43-4511-ab8b-223d8bff43a4 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:09.640497 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 913/1832] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:39:09 2022] GET /identity/v3/roles => generated 1420 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:39:09.644303 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9469b08-8a43-4511-ab8b-223d8bff43a4 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:09.663715 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9469b08-8a43-4511-ab8b-223d8bff43a4 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:09.682039 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b973bd-9896-46f4-a3e2-70e3e18c5973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:09.682835 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b973bd-9896-46f4-a3e2-70e3e18c5973 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:09.683123 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b973bd-9896-46f4-a3e2-70e3e18c5973 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:09.683537 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b973bd-9896-46f4-a3e2-70e3e18c5973 admin admin] PATH_INFO: `/v3/projects/bb675a53c68d4776b73e1586e69f4193/users/6bc7b045f6da49d5a01c206d00076039/roles/053e12af84e943e699460a9d76c2072b` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:09.715951 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589f77c4-a7d7-4782-8747-8c8544acd30d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:09.718187 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589f77c4-a7d7-4782-8747-8c8544acd30d service cinder] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:09.721806 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b973bd-9896-46f4-a3e2-70e3e18c5973 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb675a53c68d4776b73e1586e69f4193, user_id=6bc7b045f6da49d5a01c206d00076039, role_id=053e12af84e943e699460a9d76c2072b)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:09.724639 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b973bd-9896-46f4-a3e2-70e3e18c5973 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:09.752718 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 920/1833] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:39:09 2022] GET /identity/v3/auth/tokens => generated 3282 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:39:09.812336 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589f77c4-a7d7-4782-8747-8c8544acd30d service cinder] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:09.814303 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-589f77c4-a7d7-4782-8747-8c8544acd30d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:09.838177 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589f77c4-a7d7-4782-8747-8c8544acd30d service cinder] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:09.838472 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 914/1834] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:39:09 2022] PUT /identity/v3/projects/bb675a53c68d4776b73e1586e69f4193/users/6bc7b045f6da49d5a01c206d00076039/roles/053e12af84e943e699460a9d76c2072b => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:39:09.847452 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8ffc96-8363-4378-a590-68e085f76126 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:09.848108 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8ffc96-8363-4378-a590-68e085f76126 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:09.848561 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8ffc96-8363-4378-a590-68e085f76126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:09.937746 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-5d8ffc96-8363-4378-a590-68e085f76126 None None] MFA Rules not processed for user `6bc7b045f6da49d5a01c206d00076039`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:39:09.941702 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-5d8ffc96-8363-4378-a590-68e085f76126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:10.030355 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e490509-3123-48da-b603-89e212845927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:10.031020 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e490509-3123-48da-b603-89e212845927 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:10.031307 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e490509-3123-48da-b603-89e212845927 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:10.031788 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e490509-3123-48da-b603-89e212845927 admin admin] PATH_INFO: `/v3/projects/453ffafb74604ac3bcc22a765b47681d` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:10.033059 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e490509-3123-48da-b603-89e212845927 admin admin] RBAC: Authorizing `identity:delete_project(project_id=453ffafb74604ac3bcc22a765b47681d)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:10.036913 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e490509-3123-48da-b603-89e212845927 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:10.150526 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 915/1835] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:39:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 07:39:10.203804 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8ffc96-8363-4378-a590-68e085f76126 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:10.205327 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-5d8ffc96-8363-4378-a590-68e085f76126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:10.221563 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:10.223686 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8ffc96-8363-4378-a590-68e085f76126 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:10.233692 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f69ad48-042d-4614-921d-f14477b21784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:10.234561 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f69ad48-042d-4614-921d-f14477b21784 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:10.235015 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f69ad48-042d-4614-921d-f14477b21784 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:10.235515 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f69ad48-042d-4614-921d-f14477b21784 admin admin] PATH_INFO: `/v3/projects/a1063e44fa68486690d81ed819fa54d6` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:10.237515 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f69ad48-042d-4614-921d-f14477b21784 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1063e44fa68486690d81ed819fa54d6)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:10.241182 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f69ad48-042d-4614-921d-f14477b21784 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:10.253961 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-5e490509-3123-48da-b603-89e212845927 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '453ffafb74604ac3bcc22a765b47681d', 'request_id': 'req-5e490509-3123-48da-b603-89e212845927'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:10.263335 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-5e490509-3123-48da-b603-89e212845927 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '453ffafb74604ac3bcc22a765b47681d', 'request_id': 'req-5e490509-3123-48da-b603-89e212845927'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:10.384696 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-5e490509-3123-48da-b603-89e212845927 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '453ffafb74604ac3bcc22a765b47681d', 'request_id': 'req-5e490509-3123-48da-b603-89e212845927'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:10.397106 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-5e490509-3123-48da-b603-89e212845927 admin admin] The token cache is being invalidate because project 453ffafb74604ac3bcc22a765b47681d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:39:10.397488 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-5e490509-3123-48da-b603-89e212845927 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:10.399811 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-5e490509-3123-48da-b603-89e212845927 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:10.402988 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 921/1836] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:39:09 2022] DELETE /identity/v3/projects/453ffafb74604ac3bcc22a765b47681d => generated 0 bytes in 591 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 07:39:10.404434 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e490509-3123-48da-b603-89e212845927 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:10.406642 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-5e490509-3123-48da-b603-89e212845927 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:10.479802 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-8f69ad48-042d-4614-921d-f14477b21784 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1063e44fa68486690d81ed819fa54d6', 'request_id': 'req-8f69ad48-042d-4614-921d-f14477b21784'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:10.492659 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-8f69ad48-042d-4614-921d-f14477b21784 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1063e44fa68486690d81ed819fa54d6', 'request_id': 'req-8f69ad48-042d-4614-921d-f14477b21784'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:10.505800 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:10.562853 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e490509-3123-48da-b603-89e212845927 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:10.613880 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-8f69ad48-042d-4614-921d-f14477b21784 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1063e44fa68486690d81ed819fa54d6', 'request_id': 'req-8f69ad48-042d-4614-921d-f14477b21784'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:10.631862 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-8f69ad48-042d-4614-921d-f14477b21784 admin admin] The token cache is being invalidate because project a1063e44fa68486690d81ed819fa54d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:39:10.631862 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-8f69ad48-042d-4614-921d-f14477b21784 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:10.647002 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-8f69ad48-042d-4614-921d-f14477b21784 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:10.652931 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 916/1837] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:39:10 2022] DELETE /identity/v3/projects/a1063e44fa68486690d81ed819fa54d6 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:39:10.652931 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409ba8b9-f7d2-4ad4-8ceb-eeadfeee39ad None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:10.653178 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409ba8b9-f7d2-4ad4-8ceb-eeadfeee39ad None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:10.653488 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409ba8b9-f7d2-4ad4-8ceb-eeadfeee39ad None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:10.657781 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 917/1838] 10.222.0.42 () {54 vars in 921 bytes} [Thu Sep 15 07:39:10 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:39:10.659732 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0fa15a-f96b-40a9-a213-63ae747ca697 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:10.660220 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0fa15a-f96b-40a9-a213-63ae747ca697 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:10.661782 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0fa15a-f96b-40a9-a213-63ae747ca697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:10.706261 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-ed0fa15a-f96b-40a9-a213-63ae747ca697 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:39:10.707911 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-ed0fa15a-f96b-40a9-a213-63ae747ca697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:10.733855 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2295ff-bb53-4ecd-a63c-2838700acaeb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a080c7ab9b684abca6828067f2572c3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:10.734474 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2295ff-bb53-4ecd-a63c-2838700acaeb service nova] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:10.734757 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2295ff-bb53-4ecd-a63c-2838700acaeb service nova] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:10.735030 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2295ff-bb53-4ecd-a63c-2838700acaeb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:10.762873 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-7d2295ff-bb53-4ecd-a63c-2838700acaeb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:10.784962 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 918/1839] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:39:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:39:10.786568 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5222f9a9-a17c-41e5-a343-c0bce27adc06 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:10.787308 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5222f9a9-a17c-41e5-a343-c0bce27adc06 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:10.787872 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5222f9a9-a17c-41e5-a343-c0bce27adc06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:10.836506 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-5222f9a9-a17c-41e5-a343-c0bce27adc06 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:39:10.837830 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-5222f9a9-a17c-41e5-a343-c0bce27adc06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:10.876306 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 919/1840] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:39:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:39:10.879282 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feb6cdf-ae70-41e8-a7b1-6d08066de8dd None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:10.881052 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feb6cdf-ae70-41e8-a7b1-6d08066de8dd None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:10.881494 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feb6cdf-ae70-41e8-a7b1-6d08066de8dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:10.911762 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-2feb6cdf-ae70-41e8-a7b1-6d08066de8dd None None] MFA Rules not processed for user `a080c7ab9b684abca6828067f2572c3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:39:10.913975 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-2feb6cdf-ae70-41e8-a7b1-6d08066de8dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:10.926388 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 920/1841] 10.222.0.42 () {58 vars in 1018 bytes} [Thu Sep 15 07:39:10 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:39:10.928252 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df5f946-3f1a-48c6-9cf4-537af6ad865c None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:10.928642 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df5f946-3f1a-48c6-9cf4-537af6ad865c None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:10.928920 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df5f946-3f1a-48c6-9cf4-537af6ad865c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:10.948278 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2295ff-bb53-4ecd-a63c-2838700acaeb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:10.950667 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2295ff-bb53-4ecd-a63c-2838700acaeb service nova] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:10.970152 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-8df5f946-3f1a-48c6-9cf4-537af6ad865c None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:39:10.971353 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-8df5f946-3f1a-48c6-9cf4-537af6ad865c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:11.001996 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 921/1842] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:39:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:39:11.001996 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a9e12b-9087-4271-ac77-be98aa1cbec6 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:11.001996 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a9e12b-9087-4271-ac77-be98aa1cbec6 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:11.001996 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a9e12b-9087-4271-ac77-be98aa1cbec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:11.003943 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 922/1843] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:39:10 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 600 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:39:11.007236 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2295ff-bb53-4ecd-a63c-2838700acaeb service nova] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:11.009764 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-47a9e12b-9087-4271-ac77-be98aa1cbec6 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:39:11.010202 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-7d2295ff-bb53-4ecd-a63c-2838700acaeb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:11.011990 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-47a9e12b-9087-4271-ac77-be98aa1cbec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:11.023701 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 922/1844] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:39:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:39:11.023701 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a9e12b-9087-4271-ac77-be98aa1cbec6 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:11.037436 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a9e12b-9087-4271-ac77-be98aa1cbec6 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:11.052067 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e446c1-f80d-4178-be7f-a1f31bb5942d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:11.058371 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e446c1-f80d-4178-be7f-a1f31bb5942d admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:11.058371 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e446c1-f80d-4178-be7f-a1f31bb5942d admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:11.058371 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e446c1-f80d-4178-be7f-a1f31bb5942d admin admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:11.058371 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e446c1-f80d-4178-be7f-a1f31bb5942d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:39:11.058371 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e446c1-f80d-4178-be7f-a1f31bb5942d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:11.058836 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2295ff-bb53-4ecd-a63c-2838700acaeb service nova] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:11.062958 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e446c1-f80d-4178-be7f-a1f31bb5942d admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:11.071704 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c9d50a-168f-41ae-90b0-016a93a42474 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a080c7ab9b684abca6828067f2572c3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:11.072760 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c9d50a-168f-41ae-90b0-016a93a42474 service nova] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:11.073284 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c9d50a-168f-41ae-90b0-016a93a42474 service nova] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:11.073563 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c9d50a-168f-41ae-90b0-016a93a42474 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:11.085407 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c9d50a-168f-41ae-90b0-016a93a42474 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:11.088793 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c9d50a-168f-41ae-90b0-016a93a42474 service nova] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:11.095418 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 923/1845] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 15 07:39:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:39:11.096435 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e446c1-f80d-4178-be7f-a1f31bb5942d admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:11.103810 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 923/1846] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:39:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:39:11.104742 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c9d50a-168f-41ae-90b0-016a93a42474 service nova] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:11.118824 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c9d50a-168f-41ae-90b0-016a93a42474 service nova] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:11.121433 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e446c1-f80d-4178-be7f-a1f31bb5942d admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:11.125903 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be38040b-970f-40f5-93e2-50e9c5b5be6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:11.126758 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be38040b-970f-40f5-93e2-50e9c5b5be6b admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:11.127200 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be38040b-970f-40f5-93e2-50e9c5b5be6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:11.127650 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be38040b-970f-40f5-93e2-50e9c5b5be6b admin admin] PATH_INFO: `/v3/projects` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:11.128933 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be38040b-970f-40f5-93e2-50e9c5b5be6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:11.131062 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be38040b-970f-40f5-93e2-50e9c5b5be6b admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:11.136443 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37ce865-df04-4d46-b737-7931212c00e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:11.136976 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37ce865-df04-4d46-b737-7931212c00e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:11.137264 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37ce865-df04-4d46-b737-7931212c00e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:11.137541 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37ce865-df04-4d46-b737-7931212c00e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:11.138227 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37ce865-df04-4d46-b737-7931212c00e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:39:11.138534 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37ce865-df04-4d46-b737-7931212c00e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:11.141146 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37ce865-df04-4d46-b737-7931212c00e3 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:11.171692 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 924/1847] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 15 07:39:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:39:11.176287 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37ce865-df04-4d46-b737-7931212c00e3 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:11.195493 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37ce865-df04-4d46-b737-7931212c00e3 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:11.205287 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8adff7-1230-461e-ba9a-4cf34e773b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:11.205978 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8adff7-1230-461e-ba9a-4cf34e773b03 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:11.206292 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8adff7-1230-461e-ba9a-4cf34e773b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:11.206545 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8adff7-1230-461e-ba9a-4cf34e773b03 admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:11.207202 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8adff7-1230-461e-ba9a-4cf34e773b03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:11.209142 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8adff7-1230-461e-ba9a-4cf34e773b03 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:11.269427 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 924/1848] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:39:11 2022] POST /identity/v3/projects => generated 373 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:39:11.271926 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be38040b-970f-40f5-93e2-50e9c5b5be6b admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:11.336326 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 925/1849] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:39:11 2022] POST /identity/v3/projects => generated 363 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 07:39:11.340710 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8adff7-1230-461e-ba9a-4cf34e773b03 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:11.406447 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be38040b-970f-40f5-93e2-50e9c5b5be6b admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:11.458530 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8adff7-1230-461e-ba9a-4cf34e773b03 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:11.464219 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4973285e-a19d-4de3-82e2-d52562b3e402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:11.465185 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4973285e-a19d-4de3-82e2-d52562b3e402 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:11.465712 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4973285e-a19d-4de3-82e2-d52562b3e402 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:11.468237 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4973285e-a19d-4de3-82e2-d52562b3e402 admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:11.468237 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4973285e-a19d-4de3-82e2-d52562b3e402 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:11.472016 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4973285e-a19d-4de3-82e2-d52562b3e402 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:11.499209 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a56f48-2ee0-4c70-b765-35108a456d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:11.503447 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a56f48-2ee0-4c70-b765-35108a456d80 admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:11.503447 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a56f48-2ee0-4c70-b765-35108a456d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:11.503447 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a56f48-2ee0-4c70-b765-35108a456d80 admin admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:11.503447 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a56f48-2ee0-4c70-b765-35108a456d80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:11.503447 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a56f48-2ee0-4c70-b765-35108a456d80 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:11.609368 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 926/1850] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:39:11 2022] POST /identity/v3/users => generated 282 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:39:11.609368 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4973285e-a19d-4de3-82e2-d52562b3e402 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:11.610153 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 925/1851] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:39:11 2022] POST /identity/v3/users => generated 287 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:39:11.610885 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-4973285e-a19d-4de3-82e2-d52562b3e402 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:11.611890 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a56f48-2ee0-4c70-b765-35108a456d80 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:11.623704 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a56f48-2ee0-4c70-b765-35108a456d80 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:11.629080 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4973285e-a19d-4de3-82e2-d52562b3e402 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:11.630396 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639fe8bf-76f7-4ed0-b099-83b5d9c2cd6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:11.631380 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639fe8bf-76f7-4ed0-b099-83b5d9c2cd6d admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:11.631825 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639fe8bf-76f7-4ed0-b099-83b5d9c2cd6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:11.632255 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639fe8bf-76f7-4ed0-b099-83b5d9c2cd6d admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:11.633532 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639fe8bf-76f7-4ed0-b099-83b5d9c2cd6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:11.634646 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fa0b38-72c4-4cf3-afcf-e717409646e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:11.635211 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fa0b38-72c4-4cf3-afcf-e717409646e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:11.635561 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fa0b38-72c4-4cf3-afcf-e717409646e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:11.635909 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fa0b38-72c4-4cf3-afcf-e717409646e0 admin admin] PATH_INFO: `/v3/users/6bc7b045f6da49d5a01c206d00076039` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:11.636273 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639fe8bf-76f7-4ed0-b099-83b5d9c2cd6d admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:11.637292 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fa0b38-72c4-4cf3-afcf-e717409646e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6bc7b045f6da49d5a01c206d00076039)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:11.639732 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fa0b38-72c4-4cf3-afcf-e717409646e0 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:11.652091 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 926/1852] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:39:11 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:39:11.654101 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639fe8bf-76f7-4ed0-b099-83b5d9c2cd6d admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:11.668657 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639fe8bf-76f7-4ed0-b099-83b5d9c2cd6d admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:11.679149 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a8c0a4-7d38-4644-b511-6ca2c01f17e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:11.679996 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a8c0a4-7d38-4644-b511-6ca2c01f17e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:11.680496 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a8c0a4-7d38-4644-b511-6ca2c01f17e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:11.680845 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a8c0a4-7d38-4644-b511-6ca2c01f17e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:11.681499 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a8c0a4-7d38-4644-b511-6ca2c01f17e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:11.683495 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a8c0a4-7d38-4644-b511-6ca2c01f17e7 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:11.698096 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 927/1853] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:39:11 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:39:11.698096 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f795d64a-72dc-499c-94e3-c0aff6536431 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:11.698096 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f795d64a-72dc-499c-94e3-c0aff6536431 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:11.698096 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f795d64a-72dc-499c-94e3-c0aff6536431 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:11.699287 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 928/1854] 10.222.0.42 () {54 vars in 921 bytes} [Thu Sep 15 07:39:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:39:11.700229 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f795d64a-72dc-499c-94e3-c0aff6536431 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:11.720006 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f795d64a-72dc-499c-94e3-c0aff6536431 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:11.725414 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bdce4c-3593-48c3-8f05-21cba3d4eb6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:11.726053 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bdce4c-3593-48c3-8f05-21cba3d4eb6b admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:11.727082 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bdce4c-3593-48c3-8f05-21cba3d4eb6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:11.731053 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bdce4c-3593-48c3-8f05-21cba3d4eb6b admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:11.731053 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bdce4c-3593-48c3-8f05-21cba3d4eb6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:11.731053 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bdce4c-3593-48c3-8f05-21cba3d4eb6b admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:11.745878 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 929/1855] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:39:11 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:39:11.745878 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bdce4c-3593-48c3-8f05-21cba3d4eb6b admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:11.759218 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bdce4c-3593-48c3-8f05-21cba3d4eb6b admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:11.765781 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e44e90-02cb-4333-ac62-ad6c70895288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:11.766385 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e44e90-02cb-4333-ac62-ad6c70895288 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:11.766668 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e44e90-02cb-4333-ac62-ad6c70895288 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:11.766938 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e44e90-02cb-4333-ac62-ad6c70895288 admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:11.767571 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e44e90-02cb-4333-ac62-ad6c70895288 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:11.769251 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e44e90-02cb-4333-ac62-ad6c70895288 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:11.782933 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 930/1856] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:39:11 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:39:11.784356 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdba521-9336-47ac-bf0d-76a6dacd4a15 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:11.784654 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdba521-9336-47ac-bf0d-76a6dacd4a15 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:11.784936 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdba521-9336-47ac-bf0d-76a6dacd4a15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:11.806812 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-fbdba521-9336-47ac-bf0d-76a6dacd4a15 None None] MFA Rules not processed for user `a080c7ab9b684abca6828067f2572c3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:39:11.808127 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-fbdba521-9336-47ac-bf0d-76a6dacd4a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:11.884079 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 931/1857] 10.222.0.42 () {58 vars in 1018 bytes} [Thu Sep 15 07:39:11 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:39:11.884862 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdba521-9336-47ac-bf0d-76a6dacd4a15 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:11.899321 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdba521-9336-47ac-bf0d-76a6dacd4a15 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:11.904672 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7121ac8-7909-459c-a1cc-aad20c1fffd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:11.907303 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7121ac8-7909-459c-a1cc-aad20c1fffd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:11.907663 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7121ac8-7909-459c-a1cc-aad20c1fffd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:11.908015 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7121ac8-7909-459c-a1cc-aad20c1fffd1 admin admin] PATH_INFO: `/v3/projects/240918f14a0843abab182af6ff9b1e87/users/a3587236bc2e40a8a0701930579e4eed/roles/053e12af84e943e699460a9d76c2072b` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:11.935663 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-e6fa0b38-72c4-4cf3-afcf-e717409646e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6bc7b045f6da49d5a01c206d00076039'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:11.937952 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-e6fa0b38-72c4-4cf3-afcf-e717409646e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6bc7b045f6da49d5a01c206d00076039'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:11.952141 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7121ac8-7909-459c-a1cc-aad20c1fffd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=240918f14a0843abab182af6ff9b1e87, user_id=a3587236bc2e40a8a0701930579e4eed, role_id=053e12af84e943e699460a9d76c2072b)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:11.954440 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7121ac8-7909-459c-a1cc-aad20c1fffd1 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:12.074460 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-e6fa0b38-72c4-4cf3-afcf-e717409646e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6bc7b045f6da49d5a01c206d00076039'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:12.075199 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-e6fa0b38-72c4-4cf3-afcf-e717409646e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6bc7b045f6da49d5a01c206d00076039'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:12.082857 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-e6fa0b38-72c4-4cf3-afcf-e717409646e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6bc7b045f6da49d5a01c206d00076039'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:12.102057 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 927/1858] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:39:11 2022] DELETE /identity/v3/users/6bc7b045f6da49d5a01c206d00076039 => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:39:12.103125 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fa0b38-72c4-4cf3-afcf-e717409646e0 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:12.105893 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-e6fa0b38-72c4-4cf3-afcf-e717409646e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:12.121553 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 932/1859] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:39:11 2022] PUT /identity/v3/projects/240918f14a0843abab182af6ff9b1e87/users/a3587236bc2e40a8a0701930579e4eed/roles/053e12af84e943e699460a9d76c2072b => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:39:12.121553 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7121ac8-7909-459c-a1cc-aad20c1fffd1 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:12.131820 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-d7121ac8-7909-459c-a1cc-aad20c1fffd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:12.183029 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fa0b38-72c4-4cf3-afcf-e717409646e0 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:12.224830 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7121ac8-7909-459c-a1cc-aad20c1fffd1 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:12.247506 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a090243-762e-4fab-a9bc-ccad27efb24d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a080c7ab9b684abca6828067f2572c3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:12.248158 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a090243-762e-4fab-a9bc-ccad27efb24d service nova] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:12.248441 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a090243-762e-4fab-a9bc-ccad27efb24d service nova] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:12.248715 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a090243-762e-4fab-a9bc-ccad27efb24d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:12.254040 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-4a090243-762e-4fab-a9bc-ccad27efb24d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:12.259277 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3b7d12-a2c2-4afc-8ec4-06ebacf90636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:12.260006 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3b7d12-a2c2-4afc-8ec4-06ebacf90636 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:12.260293 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3b7d12-a2c2-4afc-8ec4-06ebacf90636 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:12.260731 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3b7d12-a2c2-4afc-8ec4-06ebacf90636 admin admin] PATH_INFO: `/v3/projects/c4235e5d13f849c99055557135c98747/users/1cea22003c2f4bc4a9c8c8abc116a4d9/roles/053e12af84e943e699460a9d76c2072b` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:12.270694 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a090243-762e-4fab-a9bc-ccad27efb24d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:12.272653 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a090243-762e-4fab-a9bc-ccad27efb24d service nova] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:12.284434 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 933/1860] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:39:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:39:12.285855 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3f1706-2f8a-4527-b6d7-63d9d56faa04 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:12.286281 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3f1706-2f8a-4527-b6d7-63d9d56faa04 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:12.286646 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3f1706-2f8a-4527-b6d7-63d9d56faa04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:12.288649 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3b7d12-a2c2-4afc-8ec4-06ebacf90636 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4235e5d13f849c99055557135c98747, user_id=1cea22003c2f4bc4a9c8c8abc116a4d9, role_id=053e12af84e943e699460a9d76c2072b)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:12.292516 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3b7d12-a2c2-4afc-8ec4-06ebacf90636 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:12.301297 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:12.312821 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-fe3f1706-2f8a-4527-b6d7-63d9d56faa04 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:39:12.314186 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-fe3f1706-2f8a-4527-b6d7-63d9d56faa04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:12.322754 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 934/1861] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:39:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:39:12.324069 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc410bda-920e-45f5-ab5e-32c06e64d117 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:12.324442 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc410bda-920e-45f5-ab5e-32c06e64d117 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:12.324772 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc410bda-920e-45f5-ab5e-32c06e64d117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:12.359975 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-fc410bda-920e-45f5-ab5e-32c06e64d117 None None] MFA Rules not processed for user `a3587236bc2e40a8a0701930579e4eed`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:39:12.367539 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-fc410bda-920e-45f5-ab5e-32c06e64d117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:12.420325 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 928/1862] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:39:12 2022] PUT /identity/v3/projects/c4235e5d13f849c99055557135c98747/users/1cea22003c2f4bc4a9c8c8abc116a4d9/roles/053e12af84e943e699460a9d76c2072b => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:39:12.421553 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3b7d12-a2c2-4afc-8ec4-06ebacf90636 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:12.423184 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-1b3b7d12-a2c2-4afc-8ec4-06ebacf90636 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:12.442444 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 935/1863] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:39:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:39:12.443782 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca2c2e2-24fc-4a78-bac3-412851137c40 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:12.444073 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca2c2e2-24fc-4a78-bac3-412851137c40 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:12.444348 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca2c2e2-24fc-4a78-bac3-412851137c40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:12.475964 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-0ca2c2e2-24fc-4a78-bac3-412851137c40 None None] MFA Rules not processed for user `1cea22003c2f4bc4a9c8c8abc116a4d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:39:12.477557 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-0ca2c2e2-24fc-4a78-bac3-412851137c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:12.490191 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3b7d12-a2c2-4afc-8ec4-06ebacf90636 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:12.494895 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa5d10c-7657-4006-98ad-3ed506d13871 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:12.495509 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa5d10c-7657-4006-98ad-3ed506d13871 service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:12.495790 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa5d10c-7657-4006-98ad-3ed506d13871 service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:12.496066 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa5d10c-7657-4006-98ad-3ed506d13871 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:12.553717 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 936/1864] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:39:12 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:39:12.555983 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca2c2e2-24fc-4a78-bac3-412851137c40 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:12.557514 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-0ca2c2e2-24fc-4a78-bac3-412851137c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:12.574111 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa5d10c-7657-4006-98ad-3ed506d13871 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:12.576484 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa5d10c-7657-4006-98ad-3ed506d13871 service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:12.604375 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 929/1865] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:39:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:39:12.607463 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa5d10c-7657-4006-98ad-3ed506d13871 service neutron] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:12.624897 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa5d10c-7657-4006-98ad-3ed506d13871 service neutron] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:12.640564 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd94f1b-157d-4870-ae24-99ce06e2edb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:12.641209 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd94f1b-157d-4870-ae24-99ce06e2edb9 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:12.641493 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd94f1b-157d-4870-ae24-99ce06e2edb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:12.641798 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd94f1b-157d-4870-ae24-99ce06e2edb9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:12.642584 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd94f1b-157d-4870-ae24-99ce06e2edb9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:12.646694 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd94f1b-157d-4870-ae24-99ce06e2edb9 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:12.675239 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca2c2e2-24fc-4a78-bac3-412851137c40 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:12.681719 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7f4b48-ae57-45ff-bd5a-03c1f118f6af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:12.682445 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7f4b48-ae57-45ff-bd5a-03c1f118f6af admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:12.682731 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7f4b48-ae57-45ff-bd5a-03c1f118f6af admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:12.683007 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7f4b48-ae57-45ff-bd5a-03c1f118f6af admin admin] PATH_INFO: `/v3/projects` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:12.683702 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7f4b48-ae57-45ff-bd5a-03c1f118f6af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:12.685795 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7f4b48-ae57-45ff-bd5a-03c1f118f6af admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:12.798768 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 930/1866] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:39:12 2022] POST /identity/v3/projects => generated 365 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:39:12.799812 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd94f1b-157d-4870-ae24-99ce06e2edb9 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:12.800889 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 937/1867] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:39:12 2022] POST /identity/v3/projects => generated 371 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:39:12.801818 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7f4b48-ae57-45ff-bd5a-03c1f118f6af admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:12.918398 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7f4b48-ae57-45ff-bd5a-03c1f118f6af admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:12.928196 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa3189f-7483-4538-9f50-163bb341ff08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:12.929026 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa3189f-7483-4538-9f50-163bb341ff08 admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:12.931060 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa3189f-7483-4538-9f50-163bb341ff08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:12.931491 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa3189f-7483-4538-9f50-163bb341ff08 admin admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:12.932157 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa3189f-7483-4538-9f50-163bb341ff08 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:12.934144 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa3189f-7483-4538-9f50-163bb341ff08 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:12.973361 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd94f1b-157d-4870-ae24-99ce06e2edb9 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:12.982315 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b39eca-9642-4365-965e-6194efe3bc06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:12.983293 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b39eca-9642-4365-965e-6194efe3bc06 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:12.983630 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b39eca-9642-4365-965e-6194efe3bc06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:12.983968 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b39eca-9642-4365-965e-6194efe3bc06 admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:12.986028 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b39eca-9642-4365-965e-6194efe3bc06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:12.987348 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b39eca-9642-4365-965e-6194efe3bc06 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:13.068270 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 938/1868] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:39:12 2022] POST /identity/v3/users => generated 292 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:39:13.069353 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa3189f-7483-4538-9f50-163bb341ff08 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:13.080264 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa3189f-7483-4538-9f50-163bb341ff08 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:13.084911 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f798937-6201-4384-b9a2-6c88b3d13bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:13.085514 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f798937-6201-4384-b9a2-6c88b3d13bd6 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:13.086946 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f798937-6201-4384-b9a2-6c88b3d13bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:13.087232 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f798937-6201-4384-b9a2-6c88b3d13bd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:13.087936 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f798937-6201-4384-b9a2-6c88b3d13bd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:13.089534 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f798937-6201-4384-b9a2-6c88b3d13bd6 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:13.100679 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 939/1869] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:39:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:39:13.105149 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f798937-6201-4384-b9a2-6c88b3d13bd6 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:13.115701 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f798937-6201-4384-b9a2-6c88b3d13bd6 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:13.116583 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 931/1870] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:39:12 2022] POST /identity/v3/users => generated 289 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:39:13.119850 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b39eca-9642-4365-965e-6194efe3bc06 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:13.121987 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffb546d-42bc-4ae3-80cd-81a7eaf68444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:13.122774 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb546d-42bc-4ae3-80cd-81a7eaf68444 admin admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:13.123319 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb546d-42bc-4ae3-80cd-81a7eaf68444 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:13.123524 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb546d-42bc-4ae3-80cd-81a7eaf68444 admin admin] PATH_INFO: `/v3/projects/602ac6a82f5742b98e2c0123f204301c/users/9cd2a5b64a034677afa512adb3299c36/roles/6bbb643e0d8d4bc0a9b27c26edf59216` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:13.129882 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b39eca-9642-4365-965e-6194efe3bc06 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:13.135698 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa963b75-ae28-4cf5-bdfd-135a6b15fbbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:13.135940 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa963b75-ae28-4cf5-bdfd-135a6b15fbbc admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:13.136435 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa963b75-ae28-4cf5-bdfd-135a6b15fbbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:13.136730 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa963b75-ae28-4cf5-bdfd-135a6b15fbbc admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:13.137430 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa963b75-ae28-4cf5-bdfd-135a6b15fbbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:13.139471 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa963b75-ae28-4cf5-bdfd-135a6b15fbbc admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:13.140873 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffb546d-42bc-4ae3-80cd-81a7eaf68444 admin admin] RBAC: Authorizing `identity:create_grant(project_id=602ac6a82f5742b98e2c0123f204301c, user_id=9cd2a5b64a034677afa512adb3299c36, role_id=6bbb643e0d8d4bc0a9b27c26edf59216)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:13.143683 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffb546d-42bc-4ae3-80cd-81a7eaf68444 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:13.153448 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 932/1871] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:39:13 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 07:39:13.157466 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa963b75-ae28-4cf5-bdfd-135a6b15fbbc admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:13.162765 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:13.169290 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa963b75-ae28-4cf5-bdfd-135a6b15fbbc admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:13.174490 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335931a1-f392-4ee6-9d66-b4ae570aae34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:13.175155 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335931a1-f392-4ee6-9d66-b4ae570aae34 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:13.175444 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335931a1-f392-4ee6-9d66-b4ae570aae34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:13.175725 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335931a1-f392-4ee6-9d66-b4ae570aae34 admin admin] PATH_INFO: `/v3/projects/1c95a12add4042e78493a2313ca1df27/users/2f7df4eccda3431a9eb03b8fb296a826/roles/6bbb643e0d8d4bc0a9b27c26edf59216` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:13.192750 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335931a1-f392-4ee6-9d66-b4ae570aae34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c95a12add4042e78493a2313ca1df27, user_id=2f7df4eccda3431a9eb03b8fb296a826, role_id=6bbb643e0d8d4bc0a9b27c26edf59216)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:13.196253 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335931a1-f392-4ee6-9d66-b4ae570aae34 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:13.238273 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 940/1872] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:39:13 2022] PUT /identity/v3/projects/602ac6a82f5742b98e2c0123f204301c/users/9cd2a5b64a034677afa512adb3299c36/roles/6bbb643e0d8d4bc0a9b27c26edf59216 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:39:13.242706 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffb546d-42bc-4ae3-80cd-81a7eaf68444 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:13.266463 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 933/1873] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:39:13 2022] PUT /identity/v3/projects/1c95a12add4042e78493a2313ca1df27/users/2f7df4eccda3431a9eb03b8fb296a826/roles/6bbb643e0d8d4bc0a9b27c26edf59216 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 07:39:13.271416 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335931a1-f392-4ee6-9d66-b4ae570aae34 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:13.311235 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffb546d-42bc-4ae3-80cd-81a7eaf68444 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:13.338258 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335931a1-f392-4ee6-9d66-b4ae570aae34 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:13.342805 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f834210-32f5-4dd3-a738-a235f957e661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:13.343502 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f834210-32f5-4dd3-a738-a235f957e661 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:13.343777 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f834210-32f5-4dd3-a738-a235f957e661 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:13.344049 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f834210-32f5-4dd3-a738-a235f957e661 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:13.344685 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f834210-32f5-4dd3-a738-a235f957e661 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:13.346316 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f834210-32f5-4dd3-a738-a235f957e661 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:13.359928 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 934/1874] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:39:13 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 07:39:13.361098 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f834210-32f5-4dd3-a738-a235f957e661 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:13.370875 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c94a32-9d2f-4f2b-b96c-6037fbc88574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:13.371030 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f834210-32f5-4dd3-a738-a235f957e661 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:13.371549 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c94a32-9d2f-4f2b-b96c-6037fbc88574 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:13.371911 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c94a32-9d2f-4f2b-b96c-6037fbc88574 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:13.372196 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c94a32-9d2f-4f2b-b96c-6037fbc88574 admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:13.372849 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c94a32-9d2f-4f2b-b96c-6037fbc88574 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:13.374564 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c94a32-9d2f-4f2b-b96c-6037fbc88574 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:13.376130 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf03684-7677-4294-8176-bc7edacd9859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:13.376920 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf03684-7677-4294-8176-bc7edacd9859 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:13.377269 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf03684-7677-4294-8176-bc7edacd9859 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:13.380672 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf03684-7677-4294-8176-bc7edacd9859 admin admin] PATH_INFO: `/v3/projects/1c95a12add4042e78493a2313ca1df27/users/2f7df4eccda3431a9eb03b8fb296a826/roles/6bbb643e0d8d4bc0a9b27c26edf59216` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:13.382377 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf03684-7677-4294-8176-bc7edacd9859 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c95a12add4042e78493a2313ca1df27, user_id=2f7df4eccda3431a9eb03b8fb296a826, role_id=6bbb643e0d8d4bc0a9b27c26edf59216)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:13.385308 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 941/1875] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:39:13 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 07:39:13.385656 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf03684-7677-4294-8176-bc7edacd9859 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:13.386311 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c94a32-9d2f-4f2b-b96c-6037fbc88574 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:13.387894 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-27c94a32-9d2f-4f2b-b96c-6037fbc88574 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:13.411058 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 935/1876] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:39:13 2022] PUT /identity/v3/projects/1c95a12add4042e78493a2313ca1df27/users/2f7df4eccda3431a9eb03b8fb296a826/roles/6bbb643e0d8d4bc0a9b27c26edf59216 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 07:39:13.411058 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf03684-7677-4294-8176-bc7edacd9859 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:13.415974 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c94a32-9d2f-4f2b-b96c-6037fbc88574 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:13.484570 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e5e129-148c-47e5-961f-c4dcf1613cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:13.485272 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e5e129-148c-47e5-961f-c4dcf1613cfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:13.485551 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e5e129-148c-47e5-961f-c4dcf1613cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:13.485848 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e5e129-148c-47e5-961f-c4dcf1613cfc admin admin] PATH_INFO: `/v3/projects/bb675a53c68d4776b73e1586e69f4193` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:13.486968 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e5e129-148c-47e5-961f-c4dcf1613cfc admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb675a53c68d4776b73e1586e69f4193)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:13.488873 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e5e129-148c-47e5-961f-c4dcf1613cfc admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:13.489262 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf03684-7677-4294-8176-bc7edacd9859 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:13.494851 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bdb8ee-fe0b-4529-8225-c8dcb32eff26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:13.495685 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bdb8ee-fe0b-4529-8225-c8dcb32eff26 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:13.495959 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bdb8ee-fe0b-4529-8225-c8dcb32eff26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:13.496240 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bdb8ee-fe0b-4529-8225-c8dcb32eff26 admin admin] PATH_INFO: `/v3/projects/602ac6a82f5742b98e2c0123f204301c/users/9cd2a5b64a034677afa512adb3299c36/roles/6bbb643e0d8d4bc0a9b27c26edf59216` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:13.501195 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bdb8ee-fe0b-4529-8225-c8dcb32eff26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=602ac6a82f5742b98e2c0123f204301c, user_id=9cd2a5b64a034677afa512adb3299c36, role_id=6bbb643e0d8d4bc0a9b27c26edf59216)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:13.501195 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bdb8ee-fe0b-4529-8225-c8dcb32eff26 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:13.512174 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 936/1877] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:39:13 2022] PUT /identity/v3/projects/602ac6a82f5742b98e2c0123f204301c/users/9cd2a5b64a034677afa512adb3299c36/roles/6bbb643e0d8d4bc0a9b27c26edf59216 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:39:13.513472 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ed6130-1f61-4a99-963c-2bc740a7f4d4 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:13.513908 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ed6130-1f61-4a99-963c-2bc740a7f4d4 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:13.514194 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ed6130-1f61-4a99-963c-2bc740a7f4d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:13.542645 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-46ed6130-1f61-4a99-963c-2bc740a7f4d4 None None] MFA Rules not processed for user `2f7df4eccda3431a9eb03b8fb296a826`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:39:13.543867 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-46ed6130-1f61-4a99-963c-2bc740a7f4d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:13.614092 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 937/1878] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:39:13 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:39:13.615515 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bf7c20-0eaa-45d0-b957-1223926afc25 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:13.616049 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bf7c20-0eaa-45d0-b957-1223926afc25 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:13.616333 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bf7c20-0eaa-45d0-b957-1223926afc25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:13.654346 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-08bf7c20-0eaa-45d0-b957-1223926afc25 None None] MFA Rules not processed for user `9cd2a5b64a034677afa512adb3299c36`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:39:13.654346 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-08bf7c20-0eaa-45d0-b957-1223926afc25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:13.655186 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-47e5e129-148c-47e5-961f-c4dcf1613cfc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb675a53c68d4776b73e1586e69f4193', 'request_id': 'req-47e5e129-148c-47e5-961f-c4dcf1613cfc'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:13.658486 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-47e5e129-148c-47e5-961f-c4dcf1613cfc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb675a53c68d4776b73e1586e69f4193', 'request_id': 'req-47e5e129-148c-47e5-961f-c4dcf1613cfc'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:13.719124 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 938/1879] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:39:13 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:39:13.719988 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bf7c20-0eaa-45d0-b957-1223926afc25 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:13.721280 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-08bf7c20-0eaa-45d0-b957-1223926afc25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:13.739724 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-47e5e129-148c-47e5-961f-c4dcf1613cfc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb675a53c68d4776b73e1586e69f4193', 'request_id': 'req-47e5e129-148c-47e5-961f-c4dcf1613cfc'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:13.751088 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-47e5e129-148c-47e5-961f-c4dcf1613cfc admin admin] The token cache is being invalidate because project bb675a53c68d4776b73e1586e69f4193 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:39:13.751455 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-47e5e129-148c-47e5-961f-c4dcf1613cfc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:13.752172 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-47e5e129-148c-47e5-961f-c4dcf1613cfc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:13.753740 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 942/1880] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:39:13 2022] DELETE /identity/v3/projects/bb675a53c68d4776b73e1586e69f4193 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:39:13.754514 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e5e129-148c-47e5-961f-c4dcf1613cfc admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:13.755794 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-47e5e129-148c-47e5-961f-c4dcf1613cfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:13.783539 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bf7c20-0eaa-45d0-b957-1223926afc25 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:13.787119 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a912cfde-ea38-46b4-9a17-4a57ee64918d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecf6642b7b84d3c9756bb41ab3349d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:13.787736 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a912cfde-ea38-46b4-9a17-4a57ee64918d service cinder] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:13.788015 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a912cfde-ea38-46b4-9a17-4a57ee64918d service cinder] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:13.788289 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a912cfde-ea38-46b4-9a17-4a57ee64918d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:13.790201 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-a912cfde-ea38-46b4-9a17-4a57ee64918d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:13.812215 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e5e129-148c-47e5-961f-c4dcf1613cfc admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:13.816794 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bd1db3-0350-4850-b6c8-e7c3695df445 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecf6642b7b84d3c9756bb41ab3349d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:13.817549 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bd1db3-0350-4850-b6c8-e7c3695df445 service cinder] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:13.817836 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bd1db3-0350-4850-b6c8-e7c3695df445 service cinder] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:13.818116 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bd1db3-0350-4850-b6c8-e7c3695df445 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:13.819977 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-b0bd1db3-0350-4850-b6c8-e7c3695df445 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:13.832697 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bd1db3-0350-4850-b6c8-e7c3695df445 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:13.834186 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bd1db3-0350-4850-b6c8-e7c3695df445 service cinder] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:13.844740 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 943/1881] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:39:13 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:39:13.845987 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b8c1af-3636-4183-a3c8-7475066a354c None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:13.846337 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b8c1af-3636-4183-a3c8-7475066a354c None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:13.846551 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b8c1af-3636-4183-a3c8-7475066a354c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:13.856272 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a912cfde-ea38-46b4-9a17-4a57ee64918d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:13.857735 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a912cfde-ea38-46b4-9a17-4a57ee64918d service cinder] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:13.866136 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-21b8c1af-3636-4183-a3c8-7475066a354c None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:39:13.867970 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-21b8c1af-3636-4183-a3c8-7475066a354c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:13.868861 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 939/1882] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:39:13 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:39:13.933541 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 944/1883] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:39:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:39:13.941567 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a9c82d-27a5-43f5-a9b7-1c92e1f7ce17 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:13.941903 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a9c82d-27a5-43f5-a9b7-1c92e1f7ce17 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:13.942209 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a9c82d-27a5-43f5-a9b7-1c92e1f7ce17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:13.963051 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-e2a9c82d-27a5-43f5-a9b7-1c92e1f7ce17 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:39:13.964284 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-e2a9c82d-27a5-43f5-a9b7-1c92e1f7ce17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:13.972347 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 940/1884] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:39:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:39:13.976840 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b8c1af-3636-4183-a3c8-7475066a354c None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:13.993678 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b8c1af-3636-4183-a3c8-7475066a354c None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:13.998802 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc4f812-e48c-4516-ae81-0db5d6b47c53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:13.999496 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc4f812-e48c-4516-ae81-0db5d6b47c53 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:13.999849 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc4f812-e48c-4516-ae81-0db5d6b47c53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:14.000188 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc4f812-e48c-4516-ae81-0db5d6b47c53 admin admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:14.000913 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc4f812-e48c-4516-ae81-0db5d6b47c53 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:39:14.001286 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc4f812-e48c-4516-ae81-0db5d6b47c53 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:14.003069 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc4f812-e48c-4516-ae81-0db5d6b47c53 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:14.019308 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 945/1885] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 15 07:39:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:39:14.027681 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a9c82d-27a5-43f5-a9b7-1c92e1f7ce17 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:14.040124 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a9c82d-27a5-43f5-a9b7-1c92e1f7ce17 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:14.053637 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32324176-cc9f-43c1-abc5-8eb38010ff7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:14.055069 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32324176-cc9f-43c1-abc5-8eb38010ff7d admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:14.055348 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32324176-cc9f-43c1-abc5-8eb38010ff7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:14.055628 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32324176-cc9f-43c1-abc5-8eb38010ff7d admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:14.056642 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32324176-cc9f-43c1-abc5-8eb38010ff7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:14.058575 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32324176-cc9f-43c1-abc5-8eb38010ff7d admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:14.109553 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc4f812-e48c-4516-ae81-0db5d6b47c53 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:14.111481 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-3cc4f812-e48c-4516-ae81-0db5d6b47c53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:14.133884 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc4f812-e48c-4516-ae81-0db5d6b47c53 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:14.136247 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db480415-733a-4ef8-bb1f-66f45fff1546 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecf6642b7b84d3c9756bb41ab3349d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:14.136900 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db480415-733a-4ef8-bb1f-66f45fff1546 service cinder] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:14.137187 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db480415-733a-4ef8-bb1f-66f45fff1546 service cinder] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:14.137638 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db480415-733a-4ef8-bb1f-66f45fff1546 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:14.139460 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-db480415-733a-4ef8-bb1f-66f45fff1546 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:14.177138 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 941/1886] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:39:14 2022] POST /identity/v3/projects => generated 365 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:39:14.181193 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32324176-cc9f-43c1-abc5-8eb38010ff7d admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:14.219565 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db480415-733a-4ef8-bb1f-66f45fff1546 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:14.221651 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db480415-733a-4ef8-bb1f-66f45fff1546 service cinder] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:14.232853 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 946/1887] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:39:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 07:39:14.275388 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32324176-cc9f-43c1-abc5-8eb38010ff7d admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:14.280104 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33163224-e705-4023-9468-b97ad961bd66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:14.280817 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33163224-e705-4023-9468-b97ad961bd66 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:14.281169 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33163224-e705-4023-9468-b97ad961bd66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:14.281521 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33163224-e705-4023-9468-b97ad961bd66 admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:14.282324 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33163224-e705-4023-9468-b97ad961bd66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:14.284338 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33163224-e705-4023-9468-b97ad961bd66 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:14.410642 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 942/1888] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:39:14 2022] POST /identity/v3/users => generated 283 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:39:14.417288 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db480415-733a-4ef8-bb1f-66f45fff1546 service cinder] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:14.428506 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db480415-733a-4ef8-bb1f-66f45fff1546 service cinder] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:14.434062 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a539a34-8c74-4919-ab23-914a6ac3f009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:14.434774 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a539a34-8c74-4919-ab23-914a6ac3f009 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:14.434989 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a539a34-8c74-4919-ab23-914a6ac3f009 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:14.435262 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a539a34-8c74-4919-ab23-914a6ac3f009 admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:14.435905 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a539a34-8c74-4919-ab23-914a6ac3f009 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:14.437486 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a539a34-8c74-4919-ab23-914a6ac3f009 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:14.448135 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 947/1889] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:39:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:39:14.452230 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33163224-e705-4023-9468-b97ad961bd66 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:14.463656 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33163224-e705-4023-9468-b97ad961bd66 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:14.468579 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e30878-f823-47c7-a5b0-6e50db552a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:14.469191 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e30878-f823-47c7-a5b0-6e50db552a16 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:14.469476 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e30878-f823-47c7-a5b0-6e50db552a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:14.469787 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e30878-f823-47c7-a5b0-6e50db552a16 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:14.470435 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e30878-f823-47c7-a5b0-6e50db552a16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:14.472052 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e30878-f823-47c7-a5b0-6e50db552a16 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:14.482774 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 943/1890] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:39:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:39:14.486918 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a539a34-8c74-4919-ab23-914a6ac3f009 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:14.497059 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a539a34-8c74-4919-ab23-914a6ac3f009 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:14.501590 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf4b24a-063e-441d-9d3f-22597afbe75a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:14.502285 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf4b24a-063e-441d-9d3f-22597afbe75a admin admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:14.502568 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf4b24a-063e-441d-9d3f-22597afbe75a admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:14.503415 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf4b24a-063e-441d-9d3f-22597afbe75a admin admin] PATH_INFO: `/v3/projects/4170e0145f1b444aaff40dafb6a645d6/users/d9074fedf5b346da8920438b4964fae1/roles/053e12af84e943e699460a9d76c2072b` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:14.518412 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf4b24a-063e-441d-9d3f-22597afbe75a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4170e0145f1b444aaff40dafb6a645d6, user_id=d9074fedf5b346da8920438b4964fae1, role_id=053e12af84e943e699460a9d76c2072b)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:14.521073 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf4b24a-063e-441d-9d3f-22597afbe75a admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:14.586466 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 948/1891] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:39:14 2022] PUT /identity/v3/projects/4170e0145f1b444aaff40dafb6a645d6/users/d9074fedf5b346da8920438b4964fae1/roles/053e12af84e943e699460a9d76c2072b => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:39:14.601600 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15de3f7c-b0e4-4e56-998c-9e7a24bd184e None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:14.602281 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15de3f7c-b0e4-4e56-998c-9e7a24bd184e None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:14.602708 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15de3f7c-b0e4-4e56-998c-9e7a24bd184e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:14.634158 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-15de3f7c-b0e4-4e56-998c-9e7a24bd184e None None] MFA Rules not processed for user `d9074fedf5b346da8920438b4964fae1`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:39:14.635395 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-15de3f7c-b0e4-4e56-998c-9e7a24bd184e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:14.724757 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 944/1892] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:39:14 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:39:14.732148 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf4b24a-063e-441d-9d3f-22597afbe75a admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:14.812654 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf4b24a-063e-441d-9d3f-22597afbe75a admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:14.817506 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46ef752-baee-4efd-8592-7febfbecbcbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:14.818163 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46ef752-baee-4efd-8592-7febfbecbcbe admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:14.818542 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46ef752-baee-4efd-8592-7febfbecbcbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:14.818701 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46ef752-baee-4efd-8592-7febfbecbcbe admin admin] PATH_INFO: `/v3/projects` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:14.819329 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46ef752-baee-4efd-8592-7febfbecbcbe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:14.821324 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46ef752-baee-4efd-8592-7febfbecbcbe admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:14.905861 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 949/1893] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:39:14 2022] POST /identity/v3/projects => generated 363 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:39:14.910134 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15de3f7c-b0e4-4e56-998c-9e7a24bd184e None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:14.921290 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15de3f7c-b0e4-4e56-998c-9e7a24bd184e None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:14.933062 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46ef752-baee-4efd-8592-7febfbecbcbe admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:14.936200 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-d46ef752-baee-4efd-8592-7febfbecbcbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:14.999250 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bffa28f-4b6f-420d-baff-e7ce30b81459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:15.000009 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bffa28f-4b6f-420d-baff-e7ce30b81459 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:15.000345 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bffa28f-4b6f-420d-baff-e7ce30b81459 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:15.000712 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bffa28f-4b6f-420d-baff-e7ce30b81459 admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:15.001414 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bffa28f-4b6f-420d-baff-e7ce30b81459 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:15.003411 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bffa28f-4b6f-420d-baff-e7ce30b81459 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:15.006537 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46ef752-baee-4efd-8592-7febfbecbcbe admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:15.011920 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7b3ad8-1ae2-4bdf-a426-984e3cf4ae8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:15.012632 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7b3ad8-1ae2-4bdf-a426-984e3cf4ae8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:15.012983 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7b3ad8-1ae2-4bdf-a426-984e3cf4ae8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:15.013352 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7b3ad8-1ae2-4bdf-a426-984e3cf4ae8c admin admin] PATH_INFO: `/v3/users/1cea22003c2f4bc4a9c8c8abc116a4d9` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:15.014595 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7b3ad8-1ae2-4bdf-a426-984e3cf4ae8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1cea22003c2f4bc4a9c8c8abc116a4d9)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:15.016581 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7b3ad8-1ae2-4bdf-a426-984e3cf4ae8c admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:15.123516 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 945/1894] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:39:14 2022] POST /identity/v3/users => generated 288 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:39:15.124539 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bffa28f-4b6f-420d-baff-e7ce30b81459 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:15.126270 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-1bffa28f-4b6f-420d-baff-e7ce30b81459 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:15.138062 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bffa28f-4b6f-420d-baff-e7ce30b81459 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:15.142585 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52acb139-8039-48a4-bd0c-cf7f27725819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:15.143309 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52acb139-8039-48a4-bd0c-cf7f27725819 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:15.143487 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52acb139-8039-48a4-bd0c-cf7f27725819 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:15.143726 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52acb139-8039-48a4-bd0c-cf7f27725819 admin admin] PATH_INFO: `/v3/users/7c804d651b8149f99718c576e7c187c7` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:15.144873 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52acb139-8039-48a4-bd0c-cf7f27725819 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c804d651b8149f99718c576e7c187c7)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:15.146825 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52acb139-8039-48a4-bd0c-cf7f27725819 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:15.265071 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-3e7b3ad8-1ae2-4bdf-a426-984e3cf4ae8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1cea22003c2f4bc4a9c8c8abc116a4d9'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:15.350908 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-3e7b3ad8-1ae2-4bdf-a426-984e3cf4ae8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1cea22003c2f4bc4a9c8c8abc116a4d9'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:15.351947 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-3e7b3ad8-1ae2-4bdf-a426-984e3cf4ae8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1cea22003c2f4bc4a9c8c8abc116a4d9'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:15.352677 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-3e7b3ad8-1ae2-4bdf-a426-984e3cf4ae8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1cea22003c2f4bc4a9c8c8abc116a4d9'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:15.358259 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-52acb139-8039-48a4-bd0c-cf7f27725819 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c804d651b8149f99718c576e7c187c7'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:15.358998 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-52acb139-8039-48a4-bd0c-cf7f27725819 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c804d651b8149f99718c576e7c187c7'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:15.361797 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-3e7b3ad8-1ae2-4bdf-a426-984e3cf4ae8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1cea22003c2f4bc4a9c8c8abc116a4d9'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:15.380016 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 950/1895] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:39:14 2022] DELETE /identity/v3/users/1cea22003c2f4bc4a9c8c8abc116a4d9 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 07:39:15.384683 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7b3ad8-1ae2-4bdf-a426-984e3cf4ae8c admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:15.386003 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-3e7b3ad8-1ae2-4bdf-a426-984e3cf4ae8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:15.456018 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7b3ad8-1ae2-4bdf-a426-984e3cf4ae8c admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:15.459718 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-52acb139-8039-48a4-bd0c-cf7f27725819 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c804d651b8149f99718c576e7c187c7'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:15.460385 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-52acb139-8039-48a4-bd0c-cf7f27725819 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c804d651b8149f99718c576e7c187c7'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:15.463644 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fceb28-ac00-46d2-8bea-400c2dcf0f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:15.469782 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fceb28-ac00-46d2-8bea-400c2dcf0f19 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:15.469782 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fceb28-ac00-46d2-8bea-400c2dcf0f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:15.469782 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fceb28-ac00-46d2-8bea-400c2dcf0f19 admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:15.469782 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fceb28-ac00-46d2-8bea-400c2dcf0f19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:15.469782 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fceb28-ac00-46d2-8bea-400c2dcf0f19 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:15.470607 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-52acb139-8039-48a4-bd0c-cf7f27725819 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c804d651b8149f99718c576e7c187c7'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:15.478904 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 951/1896] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:39:15 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:39:15.480230 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21537cfa-aa21-467c-8cf3-4cfd0f314315 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:15.480580 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21537cfa-aa21-467c-8cf3-4cfd0f314315 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:15.481091 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21537cfa-aa21-467c-8cf3-4cfd0f314315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:15.491348 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 946/1897] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:39:15 2022] DELETE /identity/v3/users/7c804d651b8149f99718c576e7c187c7 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:39:15.492385 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52acb139-8039-48a4-bd0c-cf7f27725819 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:15.493850 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-52acb139-8039-48a4-bd0c-cf7f27725819 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:15.501221 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-21537cfa-aa21-467c-8cf3-4cfd0f314315 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:39:15.502460 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-21537cfa-aa21-467c-8cf3-4cfd0f314315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:15.623663 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 952/1898] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:39:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:39:15.623663 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2772e1a4-7534-4837-8454-1cab33caebb7 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:15.623663 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2772e1a4-7534-4837-8454-1cab33caebb7 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:15.623663 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2772e1a4-7534-4837-8454-1cab33caebb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:15.626465 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52acb139-8039-48a4-bd0c-cf7f27725819 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:15.647001 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ea18eb-f163-4376-9d25-02b0059a64a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:15.649865 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ea18eb-f163-4376-9d25-02b0059a64a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:15.650187 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ea18eb-f163-4376-9d25-02b0059a64a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:15.650477 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ea18eb-f163-4376-9d25-02b0059a64a9 admin admin] PATH_INFO: `/v3/projects/ae7ab4f8df2d4c0582d72a66d5854848/users/4d2b1093bbee49f2ba251fbda56cd41c/roles/6bbb643e0d8d4bc0a9b27c26edf59216` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:15.683566 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ea18eb-f163-4376-9d25-02b0059a64a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae7ab4f8df2d4c0582d72a66d5854848, user_id=4d2b1093bbee49f2ba251fbda56cd41c, role_id=6bbb643e0d8d4bc0a9b27c26edf59216)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:15.684015 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-2772e1a4-7534-4837-8454-1cab33caebb7 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:39:15.685371 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-2772e1a4-7534-4837-8454-1cab33caebb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:15.687575 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ea18eb-f163-4376-9d25-02b0059a64a9 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:15.726967 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 953/1899] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:39:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:39:15.726967 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2772e1a4-7534-4837-8454-1cab33caebb7 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:15.727209 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-2772e1a4-7534-4837-8454-1cab33caebb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:15.792056 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 947/1900] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:39:15 2022] PUT /identity/v3/projects/ae7ab4f8df2d4c0582d72a66d5854848/users/4d2b1093bbee49f2ba251fbda56cd41c/roles/6bbb643e0d8d4bc0a9b27c26edf59216 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:39:15.795161 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ea18eb-f163-4376-9d25-02b0059a64a9 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:15.798451 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-80ea18eb-f163-4376-9d25-02b0059a64a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:15.859750 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2772e1a4-7534-4837-8454-1cab33caebb7 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:15.912214 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ea18eb-f163-4376-9d25-02b0059a64a9 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:15.921462 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72abda55-1541-4645-9ee8-dbd6bd4cdcf6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:15.922120 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72abda55-1541-4645-9ee8-dbd6bd4cdcf6 service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:15.922400 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72abda55-1541-4645-9ee8-dbd6bd4cdcf6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:15.923122 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72abda55-1541-4645-9ee8-dbd6bd4cdcf6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:15.956603 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:15.959564 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f460954e-7ab0-4d33-a659-f72b2010bcf7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:15.960192 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f460954e-7ab0-4d33-a659-f72b2010bcf7 service neutron] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:15.960492 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f460954e-7ab0-4d33-a659-f72b2010bcf7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:15.960847 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f460954e-7ab0-4d33-a659-f72b2010bcf7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:16.048928 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72abda55-1541-4645-9ee8-dbd6bd4cdcf6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:16.053283 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72abda55-1541-4645-9ee8-dbd6bd4cdcf6 service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:16.079595 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 948/1901] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:39:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:39:16.080366 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72abda55-1541-4645-9ee8-dbd6bd4cdcf6 service neutron] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:16.085927 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f460954e-7ab0-4d33-a659-f72b2010bcf7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:16.089464 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f460954e-7ab0-4d33-a659-f72b2010bcf7 service neutron] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:16.095861 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72abda55-1541-4645-9ee8-dbd6bd4cdcf6 service neutron] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:16.103217 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 954/1902] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:39:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:39:16.103669 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc68f9e-223d-4cc2-9512-557411e58ad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:16.104307 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc68f9e-223d-4cc2-9512-557411e58ad9 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:16.104596 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc68f9e-223d-4cc2-9512-557411e58ad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:16.104866 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc68f9e-223d-4cc2-9512-557411e58ad9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:16.105639 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc68f9e-223d-4cc2-9512-557411e58ad9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:16.107313 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc68f9e-223d-4cc2-9512-557411e58ad9 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:16.123093 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 949/1903] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:39:16 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:39:16.127160 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f460954e-7ab0-4d33-a659-f72b2010bcf7 service neutron] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:16.140997 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f460954e-7ab0-4d33-a659-f72b2010bcf7 service neutron] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:16.152599 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc8aba3-c31b-4456-ad5b-2d80065c37b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:16.153437 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc8aba3-c31b-4456-ad5b-2d80065c37b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:16.153818 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc8aba3-c31b-4456-ad5b-2d80065c37b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:16.154224 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc8aba3-c31b-4456-ad5b-2d80065c37b5 admin admin] PATH_INFO: `/v3/projects/ae7ab4f8df2d4c0582d72a66d5854848/users/4d2b1093bbee49f2ba251fbda56cd41c/roles/6bbb643e0d8d4bc0a9b27c26edf59216` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:16.157612 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc8aba3-c31b-4456-ad5b-2d80065c37b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae7ab4f8df2d4c0582d72a66d5854848, user_id=4d2b1093bbee49f2ba251fbda56cd41c, role_id=6bbb643e0d8d4bc0a9b27c26edf59216)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:16.163612 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc8aba3-c31b-4456-ad5b-2d80065c37b5 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:16.176728 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 955/1904] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:39:16 2022] PUT /identity/v3/projects/ae7ab4f8df2d4c0582d72a66d5854848/users/4d2b1093bbee49f2ba251fbda56cd41c/roles/6bbb643e0d8d4bc0a9b27c26edf59216 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:39:16.189469 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8323c3-ca49-4edb-84d4-e2f92fe68e78 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:16.189800 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8323c3-ca49-4edb-84d4-e2f92fe68e78 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:16.190082 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8323c3-ca49-4edb-84d4-e2f92fe68e78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:16.197965 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:16.222355 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-0d8323c3-ca49-4edb-84d4-e2f92fe68e78 None None] MFA Rules not processed for user `4d2b1093bbee49f2ba251fbda56cd41c`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:39:16.228823 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-0d8323c3-ca49-4edb-84d4-e2f92fe68e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:16.316377 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 950/1905] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:39:16 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:39:16.324324 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc8aba3-c31b-4456-ad5b-2d80065c37b5 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:16.326553 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-2fc8aba3-c31b-4456-ad5b-2d80065c37b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:16.405486 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc8aba3-c31b-4456-ad5b-2d80065c37b5 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:16.409451 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf12a782-391f-41f5-bd59-10e0ea89b2ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecf6642b7b84d3c9756bb41ab3349d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:16.410683 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf12a782-391f-41f5-bd59-10e0ea89b2ac service cinder] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:16.411195 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf12a782-391f-41f5-bd59-10e0ea89b2ac service cinder] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:16.411680 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf12a782-391f-41f5-bd59-10e0ea89b2ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:16.433928 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf12a782-391f-41f5-bd59-10e0ea89b2ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:16.436744 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf12a782-391f-41f5-bd59-10e0ea89b2ac service cinder] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:16.451159 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 956/1906] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:39:16 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 07:39:16.741962 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8323c3-ca49-4edb-84d4-e2f92fe68e78 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:16.769637 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8323c3-ca49-4edb-84d4-e2f92fe68e78 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:16.770857 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc8d211-c012-44c5-a6be-f4d092d77135 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecf6642b7b84d3c9756bb41ab3349d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:16.771591 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc8d211-c012-44c5-a6be-f4d092d77135 service cinder] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:16.772429 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc8d211-c012-44c5-a6be-f4d092d77135 service cinder] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:16.772429 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc8d211-c012-44c5-a6be-f4d092d77135 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:16.779376 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-1cc8d211-c012-44c5-a6be-f4d092d77135 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:16.887994 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf12a782-391f-41f5-bd59-10e0ea89b2ac service cinder] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:16.889855 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-cf12a782-391f-41f5-bd59-10e0ea89b2ac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:16.914671 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc8d211-c012-44c5-a6be-f4d092d77135 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:16.917372 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc8d211-c012-44c5-a6be-f4d092d77135 service cinder] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:16.931034 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 951/1907] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:39:16 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:39:16.973826 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc8d211-c012-44c5-a6be-f4d092d77135 service cinder] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:16.974155 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-1cc8d211-c012-44c5-a6be-f4d092d77135 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:17.095089 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf12a782-391f-41f5-bd59-10e0ea89b2ac service cinder] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:17.102216 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ebc4e3-b618-4072-b13c-f668a4357c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:17.102216 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ebc4e3-b618-4072-b13c-f668a4357c17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:17.102856 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ebc4e3-b618-4072-b13c-f668a4357c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:17.103158 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ebc4e3-b618-4072-b13c-f668a4357c17 admin admin] PATH_INFO: `/v3/projects/7717ab5e49114e9b88999bb5dcd8df45` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:17.104529 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ebc4e3-b618-4072-b13c-f668a4357c17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7717ab5e49114e9b88999bb5dcd8df45)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:17.105600 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc8d211-c012-44c5-a6be-f4d092d77135 service cinder] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:17.106521 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ebc4e3-b618-4072-b13c-f668a4357c17 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:17.111815 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe28165-1cc2-4bc0-8c88-d5f548b016c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:17.112477 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe28165-1cc2-4bc0-8c88-d5f548b016c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:17.112793 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe28165-1cc2-4bc0-8c88-d5f548b016c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:17.113073 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe28165-1cc2-4bc0-8c88-d5f548b016c6 admin admin] PATH_INFO: `/v3/projects/c4235e5d13f849c99055557135c98747` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:17.114324 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe28165-1cc2-4bc0-8c88-d5f548b016c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4235e5d13f849c99055557135c98747)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:17.116488 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe28165-1cc2-4bc0-8c88-d5f548b016c6 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:17.285340 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-f7ebc4e3-b618-4072-b13c-f668a4357c17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7717ab5e49114e9b88999bb5dcd8df45', 'request_id': 'req-f7ebc4e3-b618-4072-b13c-f668a4357c17'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:17.285765 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-bfe28165-1cc2-4bc0-8c88-d5f548b016c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4235e5d13f849c99055557135c98747', 'request_id': 'req-bfe28165-1cc2-4bc0-8c88-d5f548b016c6'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:17.291894 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-f7ebc4e3-b618-4072-b13c-f668a4357c17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7717ab5e49114e9b88999bb5dcd8df45', 'request_id': 'req-f7ebc4e3-b618-4072-b13c-f668a4357c17'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:17.292578 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-bfe28165-1cc2-4bc0-8c88-d5f548b016c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4235e5d13f849c99055557135c98747', 'request_id': 'req-bfe28165-1cc2-4bc0-8c88-d5f548b016c6'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:17.369318 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-f7ebc4e3-b618-4072-b13c-f668a4357c17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7717ab5e49114e9b88999bb5dcd8df45', 'request_id': 'req-f7ebc4e3-b618-4072-b13c-f668a4357c17'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:17.380552 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-f7ebc4e3-b618-4072-b13c-f668a4357c17 admin admin] The token cache is being invalidate because project 7717ab5e49114e9b88999bb5dcd8df45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:39:17.380892 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-f7ebc4e3-b618-4072-b13c-f668a4357c17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:17.381683 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-f7ebc4e3-b618-4072-b13c-f668a4357c17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:17.384103 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 957/1908] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:39:16 2022] DELETE /identity/v3/projects/7717ab5e49114e9b88999bb5dcd8df45 => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 07:39:17.387863 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ebc4e3-b618-4072-b13c-f668a4357c17 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:17.389219 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-f7ebc4e3-b618-4072-b13c-f668a4357c17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:17.454423 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ebc4e3-b618-4072-b13c-f668a4357c17 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:17.481416 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-bfe28165-1cc2-4bc0-8c88-d5f548b016c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4235e5d13f849c99055557135c98747', 'request_id': 'req-bfe28165-1cc2-4bc0-8c88-d5f548b016c6'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:17.498016 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-bfe28165-1cc2-4bc0-8c88-d5f548b016c6 admin admin] The token cache is being invalidate because project c4235e5d13f849c99055557135c98747 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:39:17.498352 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-bfe28165-1cc2-4bc0-8c88-d5f548b016c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:17.499025 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-bfe28165-1cc2-4bc0-8c88-d5f548b016c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:17.503993 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 952/1909] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:39:16 2022] DELETE /identity/v3/projects/c4235e5d13f849c99055557135c98747 => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 07:39:17.505151 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe28165-1cc2-4bc0-8c88-d5f548b016c6 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:17.507082 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-bfe28165-1cc2-4bc0-8c88-d5f548b016c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:17.522123 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24034b8-e501-487d-991a-bd3178f20cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:17.522773 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24034b8-e501-487d-991a-bd3178f20cec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:17.523054 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24034b8-e501-487d-991a-bd3178f20cec admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:17.523781 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24034b8-e501-487d-991a-bd3178f20cec admin admin] PATH_INFO: `/v3/users/c4c98774608f4be3a7a1be2a30d2ae33` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:17.525356 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24034b8-e501-487d-991a-bd3178f20cec admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4c98774608f4be3a7a1be2a30d2ae33)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:17.527345 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24034b8-e501-487d-991a-bd3178f20cec admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:17.573477 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe28165-1cc2-4bc0-8c88-d5f548b016c6 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:17.578180 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c135c7af-79a1-4b45-b0d8-007740897584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:17.578740 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c135c7af-79a1-4b45-b0d8-007740897584 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:17.579026 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c135c7af-79a1-4b45-b0d8-007740897584 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:17.579319 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c135c7af-79a1-4b45-b0d8-007740897584 admin admin] PATH_INFO: `/v3/users/2f7df4eccda3431a9eb03b8fb296a826` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:17.580437 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c135c7af-79a1-4b45-b0d8-007740897584 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f7df4eccda3431a9eb03b8fb296a826)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:17.582630 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c135c7af-79a1-4b45-b0d8-007740897584 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:17.720934 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-d24034b8-e501-487d-991a-bd3178f20cec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4c98774608f4be3a7a1be2a30d2ae33'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:17.794994 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-c135c7af-79a1-4b45-b0d8-007740897584 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f7df4eccda3431a9eb03b8fb296a826'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:17.795660 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-c135c7af-79a1-4b45-b0d8-007740897584 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f7df4eccda3431a9eb03b8fb296a826'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:17.808266 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-d24034b8-e501-487d-991a-bd3178f20cec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4c98774608f4be3a7a1be2a30d2ae33'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:17.809160 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-d24034b8-e501-487d-991a-bd3178f20cec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4c98774608f4be3a7a1be2a30d2ae33'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:17.810483 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-d24034b8-e501-487d-991a-bd3178f20cec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4c98774608f4be3a7a1be2a30d2ae33'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:17.819139 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-d24034b8-e501-487d-991a-bd3178f20cec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4c98774608f4be3a7a1be2a30d2ae33'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:17.838030 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 958/1910] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:39:17 2022] DELETE /identity/v3/users/c4c98774608f4be3a7a1be2a30d2ae33 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 07:39:17.877966 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-c135c7af-79a1-4b45-b0d8-007740897584 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f7df4eccda3431a9eb03b8fb296a826'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:17.878609 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-c135c7af-79a1-4b45-b0d8-007740897584 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f7df4eccda3431a9eb03b8fb296a826'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:17.893831 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-c135c7af-79a1-4b45-b0d8-007740897584 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f7df4eccda3431a9eb03b8fb296a826'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:17.922182 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 953/1911] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:39:17 2022] DELETE /identity/v3/users/2f7df4eccda3431a9eb03b8fb296a826 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 07:39:18.123072 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:18.600371 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24034b8-e501-487d-991a-bd3178f20cec admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:18.600915 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c135c7af-79a1-4b45-b0d8-007740897584 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:18.603939 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-d24034b8-e501-487d-991a-bd3178f20cec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:18.604769 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-c135c7af-79a1-4b45-b0d8-007740897584 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:18.714019 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24034b8-e501-487d-991a-bd3178f20cec admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:18.734796 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:18.790476 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c135c7af-79a1-4b45-b0d8-007740897584 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:18.798049 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f9f3c2-f84d-4be9-8b08-27a9d80bc308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:18.798906 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f9f3c2-f84d-4be9-8b08-27a9d80bc308 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:18.799364 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f9f3c2-f84d-4be9-8b08-27a9d80bc308 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:18.800844 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f9f3c2-f84d-4be9-8b08-27a9d80bc308 admin admin] PATH_INFO: `/v3/projects/c7ede89f19424467a1f33ad3b96449dc` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:18.810328 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f9f3c2-f84d-4be9-8b08-27a9d80bc308 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7ede89f19424467a1f33ad3b96449dc)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:18.812888 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f9f3c2-f84d-4be9-8b08-27a9d80bc308 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:18.827942 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab9dff7-004b-403f-809a-407789f65d74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:18.828864 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab9dff7-004b-403f-809a-407789f65d74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:18.829216 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab9dff7-004b-403f-809a-407789f65d74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:18.829560 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab9dff7-004b-403f-809a-407789f65d74 admin admin] PATH_INFO: `/v3/projects/1c95a12add4042e78493a2313ca1df27` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:18.831419 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab9dff7-004b-403f-809a-407789f65d74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c95a12add4042e78493a2313ca1df27)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:18.834497 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab9dff7-004b-403f-809a-407789f65d74 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:18.944897 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-88f9f3c2-f84d-4be9-8b08-27a9d80bc308 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7ede89f19424467a1f33ad3b96449dc', 'request_id': 'req-88f9f3c2-f84d-4be9-8b08-27a9d80bc308'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:18.952313 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-88f9f3c2-f84d-4be9-8b08-27a9d80bc308 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7ede89f19424467a1f33ad3b96449dc', 'request_id': 'req-88f9f3c2-f84d-4be9-8b08-27a9d80bc308'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:18.985666 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-9ab9dff7-004b-403f-809a-407789f65d74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c95a12add4042e78493a2313ca1df27', 'request_id': 'req-9ab9dff7-004b-403f-809a-407789f65d74'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:18.993096 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-9ab9dff7-004b-403f-809a-407789f65d74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c95a12add4042e78493a2313ca1df27', 'request_id': 'req-9ab9dff7-004b-403f-809a-407789f65d74'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:19.047247 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-88f9f3c2-f84d-4be9-8b08-27a9d80bc308 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7ede89f19424467a1f33ad3b96449dc', 'request_id': 'req-88f9f3c2-f84d-4be9-8b08-27a9d80bc308'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:19.061780 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-88f9f3c2-f84d-4be9-8b08-27a9d80bc308 admin admin] The token cache is being invalidate because project c7ede89f19424467a1f33ad3b96449dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:39:19.062172 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-88f9f3c2-f84d-4be9-8b08-27a9d80bc308 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:19.062932 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-88f9f3c2-f84d-4be9-8b08-27a9d80bc308 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:19.064807 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 954/1912] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:39:18 2022] DELETE /identity/v3/projects/c7ede89f19424467a1f33ad3b96449dc => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:39:19.070018 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141d8865-e6f6-493c-badd-0a67bec24926 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:19.071542 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141d8865-e6f6-493c-badd-0a67bec24926 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:19.072183 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141d8865-e6f6-493c-badd-0a67bec24926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:19.098781 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-141d8865-e6f6-493c-badd-0a67bec24926 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:39:19.099927 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-141d8865-e6f6-493c-badd-0a67bec24926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:19.105863 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-9ab9dff7-004b-403f-809a-407789f65d74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c95a12add4042e78493a2313ca1df27', 'request_id': 'req-9ab9dff7-004b-403f-809a-407789f65d74'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:19.120038 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-9ab9dff7-004b-403f-809a-407789f65d74 admin admin] The token cache is being invalidate because project 1c95a12add4042e78493a2313ca1df27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:39:19.120414 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-9ab9dff7-004b-403f-809a-407789f65d74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:19.121656 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-9ab9dff7-004b-403f-809a-407789f65d74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:19.124015 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 959/1913] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:39:18 2022] DELETE /identity/v3/projects/1c95a12add4042e78493a2313ca1df27 => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:39:19.129816 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f52761-abc5-4851-a3b7-29fafca0c8c1 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:19.130100 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f52761-abc5-4851-a3b7-29fafca0c8c1 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:19.130688 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f52761-abc5-4851-a3b7-29fafca0c8c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:19.156661 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-92f52761-abc5-4851-a3b7-29fafca0c8c1 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:39:19.159041 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-92f52761-abc5-4851-a3b7-29fafca0c8c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:19.170347 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 955/1914] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:39:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:39:19.179098 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeea12c-dddf-4f52-ba37-44052466f21d None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:19.179554 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeea12c-dddf-4f52-ba37-44052466f21d None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:19.180029 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeea12c-dddf-4f52-ba37-44052466f21d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:19.244533 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-bdeea12c-dddf-4f52-ba37-44052466f21d None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:39:19.246082 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-bdeea12c-dddf-4f52-ba37-44052466f21d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:19.255655 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 956/1915] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:39:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:39:19.267376 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeea12c-dddf-4f52-ba37-44052466f21d None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:19.269077 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 960/1916] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:39:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 07:39:19.277482 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb086f-d44e-4b0a-bcc6-259b70401e0c None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:19.277809 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb086f-d44e-4b0a-bcc6-259b70401e0c None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:19.278093 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb086f-d44e-4b0a-bcc6-259b70401e0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:19.279393 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeea12c-dddf-4f52-ba37-44052466f21d None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:19.284622 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f8b5cd-a9c9-4155-97f4-5f2ffa932d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:19.287900 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f8b5cd-a9c9-4155-97f4-5f2ffa932d3d admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:19.287900 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f8b5cd-a9c9-4155-97f4-5f2ffa932d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:19.287900 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f8b5cd-a9c9-4155-97f4-5f2ffa932d3d admin admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:19.287900 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f8b5cd-a9c9-4155-97f4-5f2ffa932d3d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:39:19.287900 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f8b5cd-a9c9-4155-97f4-5f2ffa932d3d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:19.288387 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f8b5cd-a9c9-4155-97f4-5f2ffa932d3d admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:19.298678 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-9dbb086f-d44e-4b0a-bcc6-259b70401e0c None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:39:19.299922 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-9dbb086f-d44e-4b0a-bcc6-259b70401e0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:19.300711 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 957/1917] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 15 07:39:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:39:19.305124 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f8b5cd-a9c9-4155-97f4-5f2ffa932d3d admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:19.308544 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 961/1918] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:39:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 07:39:19.312496 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbb086f-d44e-4b0a-bcc6-259b70401e0c None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:19.315788 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f8b5cd-a9c9-4155-97f4-5f2ffa932d3d admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:19.321514 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbb086f-d44e-4b0a-bcc6-259b70401e0c None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:19.322512 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a9d9ac-88b4-4844-9f5d-b4d4a349c4a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:19.323324 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a9d9ac-88b4-4844-9f5d-b4d4a349c4a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:19.323873 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a9d9ac-88b4-4844-9f5d-b4d4a349c4a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:19.324319 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a9d9ac-88b4-4844-9f5d-b4d4a349c4a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:19.324997 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a9d9ac-88b4-4844-9f5d-b4d4a349c4a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:19.326912 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c010d430-8797-4e52-aaf7-4664a2dd2a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:19.327160 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a9d9ac-88b4-4844-9f5d-b4d4a349c4a7 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:19.327764 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c010d430-8797-4e52-aaf7-4664a2dd2a36 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:19.328145 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c010d430-8797-4e52-aaf7-4664a2dd2a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:19.328512 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c010d430-8797-4e52-aaf7-4664a2dd2a36 admin admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:19.329238 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c010d430-8797-4e52-aaf7-4664a2dd2a36 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:39:19.329636 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c010d430-8797-4e52-aaf7-4664a2dd2a36 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:19.331330 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c010d430-8797-4e52-aaf7-4664a2dd2a36 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:19.345685 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 962/1919] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 15 07:39:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:39:19.350967 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c010d430-8797-4e52-aaf7-4664a2dd2a36 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:19.365458 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c010d430-8797-4e52-aaf7-4664a2dd2a36 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:19.371352 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a707942-51d4-4ddd-8c96-31561a899adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:19.372140 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a707942-51d4-4ddd-8c96-31561a899adf admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:19.373133 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a707942-51d4-4ddd-8c96-31561a899adf admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:19.373605 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a707942-51d4-4ddd-8c96-31561a899adf admin admin] PATH_INFO: `/v3/projects` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:19.374247 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a707942-51d4-4ddd-8c96-31561a899adf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:19.377727 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a707942-51d4-4ddd-8c96-31561a899adf admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:19.464073 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 963/1920] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:39:19 2022] POST /identity/v3/projects => generated 377 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 07:39:19.468021 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a707942-51d4-4ddd-8c96-31561a899adf admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:19.474895 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 958/1921] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:39:19 2022] POST /identity/v3/projects => generated 387 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 07:39:19.483677 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a9d9ac-88b4-4844-9f5d-b4d4a349c4a7 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:19.552816 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a707942-51d4-4ddd-8c96-31561a899adf admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:19.565439 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a9d9ac-88b4-4844-9f5d-b4d4a349c4a7 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:19.571063 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f303c15a-2a00-43ec-b5d9-c29fab5bd9cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:19.571815 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f303c15a-2a00-43ec-b5d9-c29fab5bd9cc admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:19.572200 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f303c15a-2a00-43ec-b5d9-c29fab5bd9cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:19.572607 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f303c15a-2a00-43ec-b5d9-c29fab5bd9cc admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:19.573343 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f303c15a-2a00-43ec-b5d9-c29fab5bd9cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:19.575383 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f303c15a-2a00-43ec-b5d9-c29fab5bd9cc admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:19.613743 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f58efdc-7ab5-40d6-ae8f-bed1ae991339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:19.614371 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f58efdc-7ab5-40d6-ae8f-bed1ae991339 admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:19.614782 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f58efdc-7ab5-40d6-ae8f-bed1ae991339 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:19.615068 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f58efdc-7ab5-40d6-ae8f-bed1ae991339 admin admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:19.615988 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f58efdc-7ab5-40d6-ae8f-bed1ae991339 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:19.618327 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f58efdc-7ab5-40d6-ae8f-bed1ae991339 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:19.720939 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 959/1922] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:39:19 2022] POST /identity/v3/users => generated 294 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:39:19.724953 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f303c15a-2a00-43ec-b5d9-c29fab5bd9cc admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:19.735041 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f303c15a-2a00-43ec-b5d9-c29fab5bd9cc admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:19.739593 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c384132-2df7-4834-b7d4-50e8b97a88bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:19.740179 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c384132-2df7-4834-b7d4-50e8b97a88bf admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:19.740462 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c384132-2df7-4834-b7d4-50e8b97a88bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:19.740734 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c384132-2df7-4834-b7d4-50e8b97a88bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:19.741366 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c384132-2df7-4834-b7d4-50e8b97a88bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:19.743642 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c384132-2df7-4834-b7d4-50e8b97a88bf admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:19.749804 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 964/1923] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:39:19 2022] POST /identity/v3/users => generated 289 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 07:39:19.754069 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 960/1924] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:39:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 07:39:19.756670 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f58efdc-7ab5-40d6-ae8f-bed1ae991339 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:19.757707 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c384132-2df7-4834-b7d4-50e8b97a88bf admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:19.767216 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f58efdc-7ab5-40d6-ae8f-bed1ae991339 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:19.768020 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c384132-2df7-4834-b7d4-50e8b97a88bf admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:19.772314 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637fa225-6a63-4e96-9317-d52ee2e88315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:19.772910 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637fa225-6a63-4e96-9317-d52ee2e88315 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:19.773193 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637fa225-6a63-4e96-9317-d52ee2e88315 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:19.774551 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637fa225-6a63-4e96-9317-d52ee2e88315 admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:19.776791 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637fa225-6a63-4e96-9317-d52ee2e88315 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:19.778495 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637fa225-6a63-4e96-9317-d52ee2e88315 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:19.778654 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f6701e-0225-443f-9fc7-ed7caa1765db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:19.782239 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f6701e-0225-443f-9fc7-ed7caa1765db admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:19.782578 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f6701e-0225-443f-9fc7-ed7caa1765db admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:19.782870 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f6701e-0225-443f-9fc7-ed7caa1765db admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:19.784207 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f6701e-0225-443f-9fc7-ed7caa1765db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:19.786089 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f6701e-0225-443f-9fc7-ed7caa1765db admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:19.794566 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 965/1925] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:39:19 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:39:19.798770 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637fa225-6a63-4e96-9317-d52ee2e88315 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:19.799499 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 961/1926] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:39:19 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 07:39:19.809038 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f6701e-0225-443f-9fc7-ed7caa1765db admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:19.815115 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637fa225-6a63-4e96-9317-d52ee2e88315 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:19.820209 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bfb0b6-0847-4bf4-85e1-1b2817bf969c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:19.821165 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f6701e-0225-443f-9fc7-ed7caa1765db admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:19.821658 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bfb0b6-0847-4bf4-85e1-1b2817bf969c admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:19.822082 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bfb0b6-0847-4bf4-85e1-1b2817bf969c admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:19.822355 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bfb0b6-0847-4bf4-85e1-1b2817bf969c admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:19.822995 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bfb0b6-0847-4bf4-85e1-1b2817bf969c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:19.824709 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bfb0b6-0847-4bf4-85e1-1b2817bf969c admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:19.827159 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0d2378-5154-415f-990f-1cf95c8169f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:19.828275 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0d2378-5154-415f-990f-1cf95c8169f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:19.828561 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0d2378-5154-415f-990f-1cf95c8169f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:19.828843 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0d2378-5154-415f-990f-1cf95c8169f4 admin admin] PATH_INFO: `/v3/projects/b4616631d7084b738ad3de54f9804209/users/edb6577408b5434fa5a235337942d1ee/roles/053e12af84e943e699460a9d76c2072b` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:19.842011 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 966/1927] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:39:19 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 07:39:19.843081 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bfb0b6-0847-4bf4-85e1-1b2817bf969c admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:19.848362 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0d2378-5154-415f-990f-1cf95c8169f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4616631d7084b738ad3de54f9804209, user_id=edb6577408b5434fa5a235337942d1ee, role_id=053e12af84e943e699460a9d76c2072b)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:19.851037 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0d2378-5154-415f-990f-1cf95c8169f4 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:19.853038 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bfb0b6-0847-4bf4-85e1-1b2817bf969c admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:19.857934 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff550be-2d14-4164-88e0-0d81669065d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:19.858705 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff550be-2d14-4164-88e0-0d81669065d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:19.858986 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff550be-2d14-4164-88e0-0d81669065d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:19.859258 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff550be-2d14-4164-88e0-0d81669065d5 admin admin] PATH_INFO: `/v3/projects/689f09db29af457098dbffb8b346ee27/users/47dab92a41c54285b4c1262ef5512f52/roles/053e12af84e943e699460a9d76c2072b` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:19.875185 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff550be-2d14-4164-88e0-0d81669065d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=689f09db29af457098dbffb8b346ee27, user_id=47dab92a41c54285b4c1262ef5512f52, role_id=053e12af84e943e699460a9d76c2072b)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:19.878240 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff550be-2d14-4164-88e0-0d81669065d5 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:19.940084 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 962/1928] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:39:19 2022] PUT /identity/v3/projects/b4616631d7084b738ad3de54f9804209/users/edb6577408b5434fa5a235337942d1ee/roles/053e12af84e943e699460a9d76c2072b => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 07:39:19.951044 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0266031c-cf85-4461-a344-5af03464d32e None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:19.951728 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0266031c-cf85-4461-a344-5af03464d32e None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:19.951728 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0266031c-cf85-4461-a344-5af03464d32e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:19.972798 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 967/1929] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:39:19 2022] PUT /identity/v3/projects/689f09db29af457098dbffb8b346ee27/users/47dab92a41c54285b4c1262ef5512f52/roles/053e12af84e943e699460a9d76c2072b => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 07:39:19.974449 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d7e7b3-3a75-4887-81a1-ff11a058cd75 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:19.974800 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d7e7b3-3a75-4887-81a1-ff11a058cd75 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:19.975136 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d7e7b3-3a75-4887-81a1-ff11a058cd75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:19.988888 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-0266031c-cf85-4461-a344-5af03464d32e None None] MFA Rules not processed for user `edb6577408b5434fa5a235337942d1ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:39:19.990180 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-0266031c-cf85-4461-a344-5af03464d32e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:20.003406 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-97d7e7b3-3a75-4887-81a1-ff11a058cd75 None None] MFA Rules not processed for user `47dab92a41c54285b4c1262ef5512f52`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:39:20.004584 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-97d7e7b3-3a75-4887-81a1-ff11a058cd75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:20.060647 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 963/1930] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:39:19 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 07:39:20.065108 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0266031c-cf85-4461-a344-5af03464d32e None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:20.070969 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 968/1931] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:39:19 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:39:20.075225 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d7e7b3-3a75-4887-81a1-ff11a058cd75 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:20.138831 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d7e7b3-3a75-4887-81a1-ff11a058cd75 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:20.144507 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa6bcf6-9a9e-4347-bd51-f28f9388b408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:20.145175 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0266031c-cf85-4461-a344-5af03464d32e None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:20.145315 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa6bcf6-9a9e-4347-bd51-f28f9388b408 admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:20.146906 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa6bcf6-9a9e-4347-bd51-f28f9388b408 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:20.146906 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa6bcf6-9a9e-4347-bd51-f28f9388b408 admin admin] PATH_INFO: `/v3/projects` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:20.146906 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa6bcf6-9a9e-4347-bd51-f28f9388b408 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:20.151453 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa6bcf6-9a9e-4347-bd51-f28f9388b408 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:20.151542 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaeb52a-ff8f-4f46-9f84-bcb070ff2c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:20.151542 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaeb52a-ff8f-4f46-9f84-bcb070ff2c01 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:20.151741 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaeb52a-ff8f-4f46-9f84-bcb070ff2c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:20.151945 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaeb52a-ff8f-4f46-9f84-bcb070ff2c01 admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:20.152579 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaeb52a-ff8f-4f46-9f84-bcb070ff2c01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:20.154514 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaeb52a-ff8f-4f46-9f84-bcb070ff2c01 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:20.219759 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 969/1932] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:39:20 2022] POST /identity/v3/projects => generated 375 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 07:39:20.220880 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa6bcf6-9a9e-4347-bd51-f28f9388b408 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:20.235060 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 964/1933] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:39:20 2022] POST /identity/v3/projects => generated 387 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 07:39:20.239003 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaeb52a-ff8f-4f46-9f84-bcb070ff2c01 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:20.326075 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa6bcf6-9a9e-4347-bd51-f28f9388b408 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:20.344551 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaeb52a-ff8f-4f46-9f84-bcb070ff2c01 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:20.355413 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be59b861-bf98-467e-a975-6f87f99f2bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:20.356044 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be59b861-bf98-467e-a975-6f87f99f2bcd admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:20.356324 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be59b861-bf98-467e-a975-6f87f99f2bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:20.356596 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be59b861-bf98-467e-a975-6f87f99f2bcd admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:20.357236 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be59b861-bf98-467e-a975-6f87f99f2bcd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:20.361795 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be59b861-bf98-467e-a975-6f87f99f2bcd admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:20.396051 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03850f39-d758-4943-ba15-8e213e6d9605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:20.396700 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03850f39-d758-4943-ba15-8e213e6d9605 admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:20.396982 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03850f39-d758-4943-ba15-8e213e6d9605 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:20.397250 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03850f39-d758-4943-ba15-8e213e6d9605 admin admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:20.397945 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03850f39-d758-4943-ba15-8e213e6d9605 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:20.400415 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03850f39-d758-4943-ba15-8e213e6d9605 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:20.462000 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 965/1934] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:39:20 2022] POST /identity/v3/users => generated 300 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 07:39:20.466235 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be59b861-bf98-467e-a975-6f87f99f2bcd admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:20.478055 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be59b861-bf98-467e-a975-6f87f99f2bcd admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:20.483092 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef7f678-3402-4bc7-97bf-57caff4f283c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:20.483707 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef7f678-3402-4bc7-97bf-57caff4f283c admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:20.483988 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef7f678-3402-4bc7-97bf-57caff4f283c admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:20.484257 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef7f678-3402-4bc7-97bf-57caff4f283c admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:20.484920 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef7f678-3402-4bc7-97bf-57caff4f283c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:20.486521 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef7f678-3402-4bc7-97bf-57caff4f283c admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:20.502779 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 966/1935] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:39:20 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:39:20.518339 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef7f678-3402-4bc7-97bf-57caff4f283c admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:20.533837 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef7f678-3402-4bc7-97bf-57caff4f283c admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:20.540727 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef75703-9ebd-46ff-ba2c-bae9f1351453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:20.541472 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef75703-9ebd-46ff-ba2c-bae9f1351453 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:20.541858 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef75703-9ebd-46ff-ba2c-bae9f1351453 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:20.542178 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef75703-9ebd-46ff-ba2c-bae9f1351453 admin admin] PATH_INFO: `/v3/projects/75fe11dc037a4d5f956b0835a5040545/users/106a1eebb6824f528fd015e6f536d848/roles/6bbb643e0d8d4bc0a9b27c26edf59216` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:20.548682 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 970/1936] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:39:20 2022] POST /identity/v3/users => generated 294 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:39:20.554225 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03850f39-d758-4943-ba15-8e213e6d9605 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:20.564148 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef75703-9ebd-46ff-ba2c-bae9f1351453 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75fe11dc037a4d5f956b0835a5040545, user_id=106a1eebb6824f528fd015e6f536d848, role_id=6bbb643e0d8d4bc0a9b27c26edf59216)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:20.566829 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef75703-9ebd-46ff-ba2c-bae9f1351453 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:20.572086 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03850f39-d758-4943-ba15-8e213e6d9605 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:20.576811 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320f12ab-8599-4ccd-b889-be09e0f9c8f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:20.577444 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320f12ab-8599-4ccd-b889-be09e0f9c8f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:20.577777 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320f12ab-8599-4ccd-b889-be09e0f9c8f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:20.582942 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320f12ab-8599-4ccd-b889-be09e0f9c8f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:20.582942 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320f12ab-8599-4ccd-b889-be09e0f9c8f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:20.582942 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320f12ab-8599-4ccd-b889-be09e0f9c8f8 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:20.592048 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 971/1937] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:39:20 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:39:20.596133 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320f12ab-8599-4ccd-b889-be09e0f9c8f8 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:20.606612 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320f12ab-8599-4ccd-b889-be09e0f9c8f8 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:20.611134 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f546c6-e114-4a97-b19d-3c74d7e381b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:20.611781 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f546c6-e114-4a97-b19d-3c74d7e381b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:20.612062 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f546c6-e114-4a97-b19d-3c74d7e381b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:20.612340 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f546c6-e114-4a97-b19d-3c74d7e381b7 admin admin] PATH_INFO: `/v3/projects/a3dedcc6d611402db8e9acb55be14a04/users/f16a3657eb4e46aea71b4b06b158db49/roles/6bbb643e0d8d4bc0a9b27c26edf59216` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:20.629800 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f546c6-e114-4a97-b19d-3c74d7e381b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3dedcc6d611402db8e9acb55be14a04, user_id=f16a3657eb4e46aea71b4b06b158db49, role_id=6bbb643e0d8d4bc0a9b27c26edf59216)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:20.631133 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f546c6-e114-4a97-b19d-3c74d7e381b7 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:20.677705 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 967/1938] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:39:20 2022] PUT /identity/v3/projects/75fe11dc037a4d5f956b0835a5040545/users/106a1eebb6824f528fd015e6f536d848/roles/6bbb643e0d8d4bc0a9b27c26edf59216 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 07:39:20.682751 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef75703-9ebd-46ff-ba2c-bae9f1351453 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:20.743732 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef75703-9ebd-46ff-ba2c-bae9f1351453 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:20.748718 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0ff029-782f-4f9e-8de8-6a0defbf76b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:20.749334 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0ff029-782f-4f9e-8de8-6a0defbf76b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:20.749676 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0ff029-782f-4f9e-8de8-6a0defbf76b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:20.749960 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0ff029-782f-4f9e-8de8-6a0defbf76b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:20.750604 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0ff029-782f-4f9e-8de8-6a0defbf76b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:20.752348 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0ff029-782f-4f9e-8de8-6a0defbf76b4 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:20.763149 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 968/1939] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:39:20 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 07:39:20.767160 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0ff029-782f-4f9e-8de8-6a0defbf76b4 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:20.777113 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0ff029-782f-4f9e-8de8-6a0defbf76b4 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:20.780361 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 972/1940] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:39:20 2022] PUT /identity/v3/projects/a3dedcc6d611402db8e9acb55be14a04/users/f16a3657eb4e46aea71b4b06b158db49/roles/6bbb643e0d8d4bc0a9b27c26edf59216 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 07:39:20.785823 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f546c6-e114-4a97-b19d-3c74d7e381b7 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:20.839117 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cc51a4-77ae-44d7-8d0e-7945b76929bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:20.839788 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cc51a4-77ae-44d7-8d0e-7945b76929bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:20.840079 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cc51a4-77ae-44d7-8d0e-7945b76929bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:20.840384 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cc51a4-77ae-44d7-8d0e-7945b76929bf admin admin] PATH_INFO: `/v3/projects/75fe11dc037a4d5f956b0835a5040545/users/106a1eebb6824f528fd015e6f536d848/roles/6bbb643e0d8d4bc0a9b27c26edf59216` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:20.842554 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cc51a4-77ae-44d7-8d0e-7945b76929bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=75fe11dc037a4d5f956b0835a5040545, user_id=106a1eebb6824f528fd015e6f536d848, role_id=6bbb643e0d8d4bc0a9b27c26edf59216)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:20.845470 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cc51a4-77ae-44d7-8d0e-7945b76929bf admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:20.853426 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f546c6-e114-4a97-b19d-3c74d7e381b7 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:20.858917 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 969/1941] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:39:20 2022] PUT /identity/v3/projects/75fe11dc037a4d5f956b0835a5040545/users/106a1eebb6824f528fd015e6f536d848/roles/6bbb643e0d8d4bc0a9b27c26edf59216 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 07:39:20.870834 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832ff054-23d5-429a-9ff4-bd052c892527 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:20.871125 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832ff054-23d5-429a-9ff4-bd052c892527 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:20.871401 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832ff054-23d5-429a-9ff4-bd052c892527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:20.885989 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:20.907845 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-832ff054-23d5-429a-9ff4-bd052c892527 None None] MFA Rules not processed for user `106a1eebb6824f528fd015e6f536d848`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:39:20.909059 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-832ff054-23d5-429a-9ff4-bd052c892527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:20.913505 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa717234-83d0-45bd-9765-23352b05c842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:20.914183 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa717234-83d0-45bd-9765-23352b05c842 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:20.914474 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa717234-83d0-45bd-9765-23352b05c842 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:20.914736 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa717234-83d0-45bd-9765-23352b05c842 admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:20.915372 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa717234-83d0-45bd-9765-23352b05c842 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:20.916954 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa717234-83d0-45bd-9765-23352b05c842 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:20.931021 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 973/1942] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:39:20 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 07:39:20.932218 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa717234-83d0-45bd-9765-23352b05c842 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:20.941763 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa717234-83d0-45bd-9765-23352b05c842 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:20.953201 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0401a232-3cc6-4359-b02d-ac16ba339ec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:20.953201 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0401a232-3cc6-4359-b02d-ac16ba339ec6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:20.953201 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0401a232-3cc6-4359-b02d-ac16ba339ec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:20.953201 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0401a232-3cc6-4359-b02d-ac16ba339ec6 admin admin] PATH_INFO: `/v3/projects/a3dedcc6d611402db8e9acb55be14a04/users/f16a3657eb4e46aea71b4b06b158db49/roles/6bbb643e0d8d4bc0a9b27c26edf59216` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:20.954176 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0401a232-3cc6-4359-b02d-ac16ba339ec6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3dedcc6d611402db8e9acb55be14a04, user_id=f16a3657eb4e46aea71b4b06b158db49, role_id=6bbb643e0d8d4bc0a9b27c26edf59216)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:20.954176 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0401a232-3cc6-4359-b02d-ac16ba339ec6 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:20.965532 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 974/1943] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:39:20 2022] PUT /identity/v3/projects/a3dedcc6d611402db8e9acb55be14a04/users/f16a3657eb4e46aea71b4b06b158db49/roles/6bbb643e0d8d4bc0a9b27c26edf59216 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 07:39:20.977215 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd50312-2ac7-45dd-9e47-dbd02355e5ce None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:20.977516 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd50312-2ac7-45dd-9e47-dbd02355e5ce None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:20.977866 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd50312-2ac7-45dd-9e47-dbd02355e5ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:20.987569 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 970/1944] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:39:20 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 07:39:20.999515 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832ff054-23d5-429a-9ff4-bd052c892527 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:21.000992 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-832ff054-23d5-429a-9ff4-bd052c892527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:21.005101 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-4dd50312-2ac7-45dd-9e47-dbd02355e5ce None None] MFA Rules not processed for user `f16a3657eb4e46aea71b4b06b158db49`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:39:21.006296 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-4dd50312-2ac7-45dd-9e47-dbd02355e5ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:21.060569 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832ff054-23d5-429a-9ff4-bd052c892527 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:21.074698 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 975/1945] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:39:20 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 07:39:21.087696 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd50312-2ac7-45dd-9e47-dbd02355e5ce None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:21.089438 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-4dd50312-2ac7-45dd-9e47-dbd02355e5ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:21.094438 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:21.113755 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62817e6-61c0-45de-b710-7923893ba699 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecf6642b7b84d3c9756bb41ab3349d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:21.114369 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62817e6-61c0-45de-b710-7923893ba699 service cinder] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:21.114658 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62817e6-61c0-45de-b710-7923893ba699 service cinder] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:21.114934 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62817e6-61c0-45de-b710-7923893ba699 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:21.149288 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd50312-2ac7-45dd-9e47-dbd02355e5ce None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:21.152877 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70758bc4-41e0-4d07-8bba-972a09a7e049 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecf6642b7b84d3c9756bb41ab3349d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:21.153489 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70758bc4-41e0-4d07-8bba-972a09a7e049 service cinder] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:21.153802 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70758bc4-41e0-4d07-8bba-972a09a7e049 service cinder] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:21.154074 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70758bc4-41e0-4d07-8bba-972a09a7e049 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:21.177012 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62817e6-61c0-45de-b710-7923893ba699 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:21.178478 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62817e6-61c0-45de-b710-7923893ba699 service cinder] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:21.188621 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 971/1946] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:39:20 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 07:39:21.212939 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70758bc4-41e0-4d07-8bba-972a09a7e049 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:21.214437 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70758bc4-41e0-4d07-8bba-972a09a7e049 service cinder] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:21.218317 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62817e6-61c0-45de-b710-7923893ba699 service cinder] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:21.224225 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 976/1947] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:39:21 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 07:39:21.282692 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62817e6-61c0-45de-b710-7923893ba699 service cinder] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:21.287327 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620fa996-1fa2-41bb-bc10-77960d4d0dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:21.287965 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620fa996-1fa2-41bb-bc10-77960d4d0dda admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:21.288252 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620fa996-1fa2-41bb-bc10-77960d4d0dda admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:21.288529 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620fa996-1fa2-41bb-bc10-77960d4d0dda admin admin] PATH_INFO: `/v3/users/edb6577408b5434fa5a235337942d1ee` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:21.290297 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620fa996-1fa2-41bb-bc10-77960d4d0dda admin admin] RBAC: Authorizing `identity:delete_user(user_id=edb6577408b5434fa5a235337942d1ee)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:21.292294 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620fa996-1fa2-41bb-bc10-77960d4d0dda admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:21.489568 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-620fa996-1fa2-41bb-bc10-77960d4d0dda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edb6577408b5434fa5a235337942d1ee'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:21.490192 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-620fa996-1fa2-41bb-bc10-77960d4d0dda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edb6577408b5434fa5a235337942d1ee'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:21.554354 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-620fa996-1fa2-41bb-bc10-77960d4d0dda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edb6577408b5434fa5a235337942d1ee'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:21.555860 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-620fa996-1fa2-41bb-bc10-77960d4d0dda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edb6577408b5434fa5a235337942d1ee'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:21.566638 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-620fa996-1fa2-41bb-bc10-77960d4d0dda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edb6577408b5434fa5a235337942d1ee'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:21.620119 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 972/1948] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:39:21 2022] DELETE /identity/v3/users/edb6577408b5434fa5a235337942d1ee => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 07:39:21.625682 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f011eee5-d344-4072-908e-d9e86d63f128 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:21.625933 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f011eee5-d344-4072-908e-d9e86d63f128 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:21.626301 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f011eee5-d344-4072-908e-d9e86d63f128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:21.656220 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-f011eee5-d344-4072-908e-d9e86d63f128 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:39:21.657551 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-f011eee5-d344-4072-908e-d9e86d63f128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:21.726192 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 977/1949] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:39:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:39:21.740237 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620fa996-1fa2-41bb-bc10-77960d4d0dda admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:21.741938 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-620fa996-1fa2-41bb-bc10-77960d4d0dda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:21.806090 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620fa996-1fa2-41bb-bc10-77960d4d0dda admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:21.809695 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd005cbf-cc0c-414b-bfe4-6ed9f8b4a193 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:21.810350 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd005cbf-cc0c-414b-bfe4-6ed9f8b4a193 service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:21.810630 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd005cbf-cc0c-414b-bfe4-6ed9f8b4a193 service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:21.810901 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd005cbf-cc0c-414b-bfe4-6ed9f8b4a193 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:21.832496 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd005cbf-cc0c-414b-bfe4-6ed9f8b4a193 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:21.833998 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd005cbf-cc0c-414b-bfe4-6ed9f8b4a193 service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:21.846474 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 973/1950] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:39:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:39:21.921627 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:22.003467 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:22.279340 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f011eee5-d344-4072-908e-d9e86d63f128 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:22.280750 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-f011eee5-d344-4072-908e-d9e86d63f128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:22.293884 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f011eee5-d344-4072-908e-d9e86d63f128 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:22.298815 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b4afd3-f830-40ec-a1b4-15506a61696f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:22.299537 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b4afd3-f830-40ec-a1b4-15506a61696f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:22.299770 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b4afd3-f830-40ec-a1b4-15506a61696f admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:22.300045 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b4afd3-f830-40ec-a1b4-15506a61696f admin admin] PATH_INFO: `/v3/projects/b4616631d7084b738ad3de54f9804209` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:22.301601 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b4afd3-f830-40ec-a1b4-15506a61696f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4616631d7084b738ad3de54f9804209)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:22.303606 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b4afd3-f830-40ec-a1b4-15506a61696f admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:22.448393 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-07b4afd3-f830-40ec-a1b4-15506a61696f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4616631d7084b738ad3de54f9804209', 'request_id': 'req-07b4afd3-f830-40ec-a1b4-15506a61696f'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:22.456365 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-07b4afd3-f830-40ec-a1b4-15506a61696f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4616631d7084b738ad3de54f9804209', 'request_id': 'req-07b4afd3-f830-40ec-a1b4-15506a61696f'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:22.482359 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:22.534625 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-07b4afd3-f830-40ec-a1b4-15506a61696f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4616631d7084b738ad3de54f9804209', 'request_id': 'req-07b4afd3-f830-40ec-a1b4-15506a61696f'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:22.547139 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-07b4afd3-f830-40ec-a1b4-15506a61696f admin admin] The token cache is being invalidate because project b4616631d7084b738ad3de54f9804209 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:39:22.547438 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-07b4afd3-f830-40ec-a1b4-15506a61696f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:22.548128 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-07b4afd3-f830-40ec-a1b4-15506a61696f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:22.549878 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 978/1951] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:39:22 2022] DELETE /identity/v3/projects/b4616631d7084b738ad3de54f9804209 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:39:22.553765 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd005cbf-cc0c-414b-bfe4-6ed9f8b4a193 service neutron] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:22.565737 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd005cbf-cc0c-414b-bfe4-6ed9f8b4a193 service neutron] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:22.624387 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f225d5-7090-493f-af1a-7989680f6500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:22.625028 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f225d5-7090-493f-af1a-7989680f6500 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:22.625341 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f225d5-7090-493f-af1a-7989680f6500 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:22.625648 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f225d5-7090-493f-af1a-7989680f6500 admin admin] PATH_INFO: `/v3/users/106a1eebb6824f528fd015e6f536d848` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:22.626771 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f225d5-7090-493f-af1a-7989680f6500 admin admin] RBAC: Authorizing `identity:delete_user(user_id=106a1eebb6824f528fd015e6f536d848)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:22.628705 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f225d5-7090-493f-af1a-7989680f6500 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:22.797409 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:22.802386 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-61f225d5-7090-493f-af1a-7989680f6500 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '106a1eebb6824f528fd015e6f536d848'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:22.803250 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-61f225d5-7090-493f-af1a-7989680f6500 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '106a1eebb6824f528fd015e6f536d848'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:22.892843 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-61f225d5-7090-493f-af1a-7989680f6500 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '106a1eebb6824f528fd015e6f536d848'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:22.893561 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-61f225d5-7090-493f-af1a-7989680f6500 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '106a1eebb6824f528fd015e6f536d848'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:22.902221 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-61f225d5-7090-493f-af1a-7989680f6500 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '106a1eebb6824f528fd015e6f536d848'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:22.922528 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 974/1952] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:39:22 2022] DELETE /identity/v3/users/106a1eebb6824f528fd015e6f536d848 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:39:22.999630 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b4afd3-f830-40ec-a1b4-15506a61696f admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:23.001132 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-07b4afd3-f830-40ec-a1b4-15506a61696f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:23.031973 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b4afd3-f830-40ec-a1b4-15506a61696f admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:23.097509 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:23.134481 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2c558a-08fc-491e-8aa2-60a5e59811aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:23.135157 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2c558a-08fc-491e-8aa2-60a5e59811aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:23.135436 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2c558a-08fc-491e-8aa2-60a5e59811aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:23.135750 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2c558a-08fc-491e-8aa2-60a5e59811aa admin admin] PATH_INFO: `/v3/users/d33704412b3245fc9b31395f0611483f` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:23.136957 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2c558a-08fc-491e-8aa2-60a5e59811aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=d33704412b3245fc9b31395f0611483f)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:23.150849 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2c558a-08fc-491e-8aa2-60a5e59811aa admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:23.379729 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-6f2c558a-08fc-491e-8aa2-60a5e59811aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd33704412b3245fc9b31395f0611483f'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:23.483803 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-6f2c558a-08fc-491e-8aa2-60a5e59811aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd33704412b3245fc9b31395f0611483f'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:23.487195 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-6f2c558a-08fc-491e-8aa2-60a5e59811aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd33704412b3245fc9b31395f0611483f'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:23.487195 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-6f2c558a-08fc-491e-8aa2-60a5e59811aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd33704412b3245fc9b31395f0611483f'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:23.491591 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f225d5-7090-493f-af1a-7989680f6500 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:23.494057 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-61f225d5-7090-493f-af1a-7989680f6500 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:23.499094 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-6f2c558a-08fc-491e-8aa2-60a5e59811aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd33704412b3245fc9b31395f0611483f'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:23.515892 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f225d5-7090-493f-af1a-7989680f6500 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:23.520373 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c860448d-b074-450c-8483-3b509d8b8e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:23.521162 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c860448d-b074-450c-8483-3b509d8b8e0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:23.521510 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c860448d-b074-450c-8483-3b509d8b8e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:23.522017 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c860448d-b074-450c-8483-3b509d8b8e0d admin admin] PATH_INFO: `/v3/projects/75fe11dc037a4d5f956b0835a5040545` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:23.523225 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c860448d-b074-450c-8483-3b509d8b8e0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=75fe11dc037a4d5f956b0835a5040545)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:23.525548 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c860448d-b074-450c-8483-3b509d8b8e0d admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:23.544153 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 979/1953] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:39:22 2022] DELETE /identity/v3/users/d33704412b3245fc9b31395f0611483f => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 07:39:23.544153 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702f4109-a9f1-4aeb-be4f-905ee9cbdd10 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:23.544153 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702f4109-a9f1-4aeb-be4f-905ee9cbdd10 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:23.544153 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702f4109-a9f1-4aeb-be4f-905ee9cbdd10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:23.566081 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-702f4109-a9f1-4aeb-be4f-905ee9cbdd10 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:39:23.567281 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-702f4109-a9f1-4aeb-be4f-905ee9cbdd10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:23.637121 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 980/1954] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:39:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:39:23.644086 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:23.653888 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702f4109-a9f1-4aeb-be4f-905ee9cbdd10 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:23.653888 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-702f4109-a9f1-4aeb-be4f-905ee9cbdd10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:23.678542 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-c860448d-b074-450c-8483-3b509d8b8e0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75fe11dc037a4d5f956b0835a5040545', 'request_id': 'req-c860448d-b074-450c-8483-3b509d8b8e0d'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:23.690500 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-c860448d-b074-450c-8483-3b509d8b8e0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75fe11dc037a4d5f956b0835a5040545', 'request_id': 'req-c860448d-b074-450c-8483-3b509d8b8e0d'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:23.737373 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702f4109-a9f1-4aeb-be4f-905ee9cbdd10 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:23.757453 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-c860448d-b074-450c-8483-3b509d8b8e0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75fe11dc037a4d5f956b0835a5040545', 'request_id': 'req-c860448d-b074-450c-8483-3b509d8b8e0d'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:23.770453 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-c860448d-b074-450c-8483-3b509d8b8e0d admin admin] The token cache is being invalidate because project 75fe11dc037a4d5f956b0835a5040545 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:39:23.770981 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-c860448d-b074-450c-8483-3b509d8b8e0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:23.771715 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-c860448d-b074-450c-8483-3b509d8b8e0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:23.774699 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 975/1955] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:39:23 2022] DELETE /identity/v3/projects/75fe11dc037a4d5f956b0835a5040545 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 07:39:23.779385 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b4002f-6156-4560-a356-34fa00140f43 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:23.780589 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b4002f-6156-4560-a356-34fa00140f43 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:23.780883 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b4002f-6156-4560-a356-34fa00140f43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:23.793416 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab0c6e9-52c7-464f-8e4e-1a21d4cb8a05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:23.794174 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab0c6e9-52c7-464f-8e4e-1a21d4cb8a05 service neutron] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:23.794457 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab0c6e9-52c7-464f-8e4e-1a21d4cb8a05 service neutron] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:23.794725 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab0c6e9-52c7-464f-8e4e-1a21d4cb8a05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:23.796723 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-6ab0c6e9-52c7-464f-8e4e-1a21d4cb8a05 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:23.802996 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-74b4002f-6156-4560-a356-34fa00140f43 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:39:23.804196 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-74b4002f-6156-4560-a356-34fa00140f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:23.861267 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:23.888316 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 976/1956] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:39:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 07:39:23.896301 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a128dc-d775-4989-8380-ee0678a3f719 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:23.896576 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a128dc-d775-4989-8380-ee0678a3f719 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:23.896847 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a128dc-d775-4989-8380-ee0678a3f719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:23.922746 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab0c6e9-52c7-464f-8e4e-1a21d4cb8a05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:23.924405 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab0c6e9-52c7-464f-8e4e-1a21d4cb8a05 service neutron] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:23.927243 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-41a128dc-d775-4989-8380-ee0678a3f719 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:39:23.928523 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-41a128dc-d775-4989-8380-ee0678a3f719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:23.937420 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 981/1957] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:39:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:39:23.938838 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 977/1958] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:39:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 07:39:23.950719 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab0c6e9-52c7-464f-8e4e-1a21d4cb8a05 service neutron] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:23.966798 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab0c6e9-52c7-464f-8e4e-1a21d4cb8a05 service neutron] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:23.974504 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4475c6-7b0d-49d3-9ef8-0634618c0c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:23.975689 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4475c6-7b0d-49d3-9ef8-0634618c0c96 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:23.976023 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4475c6-7b0d-49d3-9ef8-0634618c0c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:23.976335 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4475c6-7b0d-49d3-9ef8-0634618c0c96 admin admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:23.977511 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4475c6-7b0d-49d3-9ef8-0634618c0c96 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:39:23.977846 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4475c6-7b0d-49d3-9ef8-0634618c0c96 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:23.979538 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4475c6-7b0d-49d3-9ef8-0634618c0c96 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:23.998600 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 982/1959] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 15 07:39:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:39:24.005130 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a128dc-d775-4989-8380-ee0678a3f719 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:24.016279 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a128dc-d775-4989-8380-ee0678a3f719 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:24.039485 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876c5136-9c84-44cd-b48e-b17a4b2e1dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:24.040886 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876c5136-9c84-44cd-b48e-b17a4b2e1dc9 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:24.041636 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876c5136-9c84-44cd-b48e-b17a4b2e1dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:24.041751 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876c5136-9c84-44cd-b48e-b17a4b2e1dc9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:24.042375 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876c5136-9c84-44cd-b48e-b17a4b2e1dc9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:24.044313 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876c5136-9c84-44cd-b48e-b17a4b2e1dc9 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:24.166644 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 978/1960] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:39:24 2022] POST /identity/v3/projects => generated 387 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:39:24.170670 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4475c6-7b0d-49d3-9ef8-0634618c0c96 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:24.188607 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4475c6-7b0d-49d3-9ef8-0634618c0c96 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:24.238782 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:24.321172 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c9054a-8a1e-4bf7-88b6-581f334fe913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:24.321934 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c9054a-8a1e-4bf7-88b6-581f334fe913 admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:24.322384 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c9054a-8a1e-4bf7-88b6-581f334fe913 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:24.322584 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c9054a-8a1e-4bf7-88b6-581f334fe913 admin admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:24.323244 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c9054a-8a1e-4bf7-88b6-581f334fe913 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:24.338534 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c9054a-8a1e-4bf7-88b6-581f334fe913 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:24.454150 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 983/1961] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:39:24 2022] POST /identity/v3/users => generated 294 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 07:39:24.455088 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876c5136-9c84-44cd-b48e-b17a4b2e1dc9 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:24.475776 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876c5136-9c84-44cd-b48e-b17a4b2e1dc9 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:24.490727 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f68d463-cbf3-49f9-acff-2587e533a284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:24.492616 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f68d463-cbf3-49f9-acff-2587e533a284 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:24.493909 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f68d463-cbf3-49f9-acff-2587e533a284 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:24.493909 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f68d463-cbf3-49f9-acff-2587e533a284 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:24.494088 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f68d463-cbf3-49f9-acff-2587e533a284 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:24.495889 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f68d463-cbf3-49f9-acff-2587e533a284 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:24.511192 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 979/1962] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:39:24 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:39:24.512174 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c9054a-8a1e-4bf7-88b6-581f334fe913 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:24.526455 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c9054a-8a1e-4bf7-88b6-581f334fe913 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:24.529311 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:24.538310 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3894fd3-808c-4597-8b00-ca8241c887f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:24.540143 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3894fd3-808c-4597-8b00-ca8241c887f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:24.540143 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3894fd3-808c-4597-8b00-ca8241c887f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:24.540741 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3894fd3-808c-4597-8b00-ca8241c887f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:24.541456 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3894fd3-808c-4597-8b00-ca8241c887f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:24.543573 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3894fd3-808c-4597-8b00-ca8241c887f6 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:24.558395 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 984/1963] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:39:24 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 07:39:24.560904 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f68d463-cbf3-49f9-acff-2587e533a284 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:24.588169 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3894fd3-808c-4597-8b00-ca8241c887f6 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:24.592569 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f68d463-cbf3-49f9-acff-2587e533a284 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:24.594991 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-d3894fd3-808c-4597-8b00-ca8241c887f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:24.598201 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b87c098-7b71-43b6-8706-60c475f4aa86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:24.598869 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b87c098-7b71-43b6-8706-60c475f4aa86 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:24.599150 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b87c098-7b71-43b6-8706-60c475f4aa86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:24.599427 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b87c098-7b71-43b6-8706-60c475f4aa86 admin admin] PATH_INFO: `/v3/projects/e913941efffd4e1a9ac45c05a0a9b086/users/bd4a548a417945519b5293738391c676/roles/053e12af84e943e699460a9d76c2072b` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:24.612838 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3894fd3-808c-4597-8b00-ca8241c887f6 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:24.615693 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301a835d-08b6-4d5f-b143-206b8c56c3d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:24.616338 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301a835d-08b6-4d5f-b143-206b8c56c3d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:24.616924 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301a835d-08b6-4d5f-b143-206b8c56c3d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:24.616924 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301a835d-08b6-4d5f-b143-206b8c56c3d1 admin admin] PATH_INFO: `/v3/projects/673b70ad99494879b805f96da622b1f4` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:24.618172 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301a835d-08b6-4d5f-b143-206b8c56c3d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=673b70ad99494879b805f96da622b1f4)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:24.620317 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301a835d-08b6-4d5f-b143-206b8c56c3d1 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:24.622933 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b87c098-7b71-43b6-8706-60c475f4aa86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e913941efffd4e1a9ac45c05a0a9b086, user_id=bd4a548a417945519b5293738391c676, role_id=053e12af84e943e699460a9d76c2072b)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:24.626799 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b87c098-7b71-43b6-8706-60c475f4aa86 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:24.741067 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 980/1964] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:39:24 2022] PUT /identity/v3/projects/e913941efffd4e1a9ac45c05a0a9b086/users/bd4a548a417945519b5293738391c676/roles/053e12af84e943e699460a9d76c2072b => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:39:24.752830 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ce53d1-1680-4867-a2ca-563970ab8990 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:24.753129 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ce53d1-1680-4867-a2ca-563970ab8990 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:24.753422 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ce53d1-1680-4867-a2ca-563970ab8990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:24.813081 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-301a835d-08b6-4d5f-b143-206b8c56c3d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '673b70ad99494879b805f96da622b1f4', 'request_id': 'req-301a835d-08b6-4d5f-b143-206b8c56c3d1'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:24.819928 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-35ce53d1-1680-4867-a2ca-563970ab8990 None None] MFA Rules not processed for user `bd4a548a417945519b5293738391c676`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:39:24.820867 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-301a835d-08b6-4d5f-b143-206b8c56c3d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '673b70ad99494879b805f96da622b1f4', 'request_id': 'req-301a835d-08b6-4d5f-b143-206b8c56c3d1'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:24.821989 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-35ce53d1-1680-4867-a2ca-563970ab8990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:24.906316 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-301a835d-08b6-4d5f-b143-206b8c56c3d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '673b70ad99494879b805f96da622b1f4', 'request_id': 'req-301a835d-08b6-4d5f-b143-206b8c56c3d1'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:24.908655 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 981/1965] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:39:24 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 07:39:24.912751 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ce53d1-1680-4867-a2ca-563970ab8990 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:24.923783 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-301a835d-08b6-4d5f-b143-206b8c56c3d1 admin admin] The token cache is being invalidate because project 673b70ad99494879b805f96da622b1f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:39:24.924159 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-301a835d-08b6-4d5f-b143-206b8c56c3d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:24.924806 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-301a835d-08b6-4d5f-b143-206b8c56c3d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:24.927744 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 985/1966] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:39:24 2022] DELETE /identity/v3/projects/673b70ad99494879b805f96da622b1f4 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:39:24.933964 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301a835d-08b6-4d5f-b143-206b8c56c3d1 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:24.937212 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-301a835d-08b6-4d5f-b143-206b8c56c3d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:25.020243 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ce53d1-1680-4867-a2ca-563970ab8990 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:25.027062 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51393689-b438-4d31-aa2c-a5df3abb930a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:25.028407 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51393689-b438-4d31-aa2c-a5df3abb930a admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:25.028754 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51393689-b438-4d31-aa2c-a5df3abb930a admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:25.029176 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51393689-b438-4d31-aa2c-a5df3abb930a admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:25.029857 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51393689-b438-4d31-aa2c-a5df3abb930a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:25.035637 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51393689-b438-4d31-aa2c-a5df3abb930a admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:25.040990 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301a835d-08b6-4d5f-b143-206b8c56c3d1 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:25.047501 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e57aa7-e036-4ac5-a6c8-4341b1269188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:25.048311 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e57aa7-e036-4ac5-a6c8-4341b1269188 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:25.048602 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e57aa7-e036-4ac5-a6c8-4341b1269188 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:25.049386 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e57aa7-e036-4ac5-a6c8-4341b1269188 admin admin] PATH_INFO: `/v3/users/d359f1820758445da420a679d3ad7544` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:25.053619 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e57aa7-e036-4ac5-a6c8-4341b1269188 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d359f1820758445da420a679d3ad7544)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:25.053619 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e57aa7-e036-4ac5-a6c8-4341b1269188 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:25.095132 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=92,keystone:INSERT=6,keystone:DELETE=2 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:25.169000 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 982/1967] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:39:24 2022] POST /identity/v3/projects => generated 385 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:39:25.170094 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51393689-b438-4d31-aa2c-a5df3abb930a admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:25.171966 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-51393689-b438-4d31-aa2c-a5df3abb930a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:25.248397 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51393689-b438-4d31-aa2c-a5df3abb930a admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:25.248397 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9655fcc8-509f-428c-965e-737f18546cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:25.248397 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9655fcc8-509f-428c-965e-737f18546cd4 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:25.248820 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9655fcc8-509f-428c-965e-737f18546cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:25.248820 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9655fcc8-509f-428c-965e-737f18546cd4 admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:25.249522 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9655fcc8-509f-428c-965e-737f18546cd4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:25.251506 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9655fcc8-509f-428c-965e-737f18546cd4 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:25.307033 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-50e57aa7-e036-4ac5-a6c8-4341b1269188 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd359f1820758445da420a679d3ad7544'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:25.375240 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-50e57aa7-e036-4ac5-a6c8-4341b1269188 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd359f1820758445da420a679d3ad7544'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:25.378966 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-50e57aa7-e036-4ac5-a6c8-4341b1269188 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd359f1820758445da420a679d3ad7544'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:25.378966 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-50e57aa7-e036-4ac5-a6c8-4341b1269188 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd359f1820758445da420a679d3ad7544'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:25.384346 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 983/1968] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:39:25 2022] POST /identity/v3/users => generated 293 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:39:25.385802 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9655fcc8-509f-428c-965e-737f18546cd4 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:25.391592 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-9655fcc8-509f-428c-965e-737f18546cd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:25.401991 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-50e57aa7-e036-4ac5-a6c8-4341b1269188 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd359f1820758445da420a679d3ad7544'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:25.423041 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9655fcc8-509f-428c-965e-737f18546cd4 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:25.428887 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f823a86-f03e-49ac-ac03-5192e5e051fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:25.429274 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f823a86-f03e-49ac-ac03-5192e5e051fb admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:25.429563 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f823a86-f03e-49ac-ac03-5192e5e051fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:25.430605 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f823a86-f03e-49ac-ac03-5192e5e051fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:25.431346 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f823a86-f03e-49ac-ac03-5192e5e051fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:25.436322 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f823a86-f03e-49ac-ac03-5192e5e051fb admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:25.449438 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 986/1969] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:39:24 2022] DELETE /identity/v3/users/d359f1820758445da420a679d3ad7544 => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:39:25.460131 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 984/1970] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:39:25 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:39:25.463418 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e57aa7-e036-4ac5-a6c8-4341b1269188 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:25.572916 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e57aa7-e036-4ac5-a6c8-4341b1269188 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:25.578097 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a1e376-b37a-40d6-9fe1-9cea25559610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:25.580968 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a1e376-b37a-40d6-9fe1-9cea25559610 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:25.581270 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a1e376-b37a-40d6-9fe1-9cea25559610 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:25.581551 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a1e376-b37a-40d6-9fe1-9cea25559610 admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:25.582647 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a1e376-b37a-40d6-9fe1-9cea25559610 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:25.584200 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a1e376-b37a-40d6-9fe1-9cea25559610 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:25.599380 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 987/1971] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:39:25 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:39:25.600630 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f823a86-f03e-49ac-ac03-5192e5e051fb admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:25.612650 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f823a86-f03e-49ac-ac03-5192e5e051fb admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:25.617378 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7a0592-a114-4364-9807-6ba6e5fc2163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:25.619275 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7a0592-a114-4364-9807-6ba6e5fc2163 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:25.619559 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7a0592-a114-4364-9807-6ba6e5fc2163 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:25.619841 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7a0592-a114-4364-9807-6ba6e5fc2163 admin admin] PATH_INFO: `/v3/projects/9321b7dd6ea642bf8cf77cc164a4f0b8/users/13a80e491b31441980abeed766bcb1de/roles/053e12af84e943e699460a9d76c2072b` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:25.635887 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7a0592-a114-4364-9807-6ba6e5fc2163 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9321b7dd6ea642bf8cf77cc164a4f0b8, user_id=13a80e491b31441980abeed766bcb1de, role_id=053e12af84e943e699460a9d76c2072b)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:25.638659 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7a0592-a114-4364-9807-6ba6e5fc2163 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:25.735535 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 985/1972] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:39:25 2022] PUT /identity/v3/projects/9321b7dd6ea642bf8cf77cc164a4f0b8/users/13a80e491b31441980abeed766bcb1de/roles/053e12af84e943e699460a9d76c2072b => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:39:25.748336 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aa36a9-ca65-483d-8e78-1a018a063aaf None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:25.748997 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aa36a9-ca65-483d-8e78-1a018a063aaf None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:25.749472 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aa36a9-ca65-483d-8e78-1a018a063aaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:25.788409 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-f6aa36a9-ca65-483d-8e78-1a018a063aaf None None] MFA Rules not processed for user `13a80e491b31441980abeed766bcb1de`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:39:25.790287 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-f6aa36a9-ca65-483d-8e78-1a018a063aaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:25.877471 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 988/1973] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:39:25 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:39:25.882113 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7a0592-a114-4364-9807-6ba6e5fc2163 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:25.959199 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aa36a9-ca65-483d-8e78-1a018a063aaf None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:25.964012 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-f6aa36a9-ca65-483d-8e78-1a018a063aaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:25.967011 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7a0592-a114-4364-9807-6ba6e5fc2163 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:25.971848 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc20795-d344-42e3-8648-e95556437793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:25.972608 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc20795-d344-42e3-8648-e95556437793 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:25.978772 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc20795-d344-42e3-8648-e95556437793 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:25.979231 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc20795-d344-42e3-8648-e95556437793 admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:25.979957 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc20795-d344-42e3-8648-e95556437793 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:25.982546 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc20795-d344-42e3-8648-e95556437793 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:25.986406 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aa36a9-ca65-483d-8e78-1a018a063aaf None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:25.997562 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39357d11-f01a-47a0-9939-b0ed8e394853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:25.999016 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39357d11-f01a-47a0-9939-b0ed8e394853 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:25.999431 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39357d11-f01a-47a0-9939-b0ed8e394853 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:25.999845 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39357d11-f01a-47a0-9939-b0ed8e394853 admin admin] PATH_INFO: `/v3/projects/a2f8eb7aec3c436fbfb1a0eb8d3df0b6` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:26.001282 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39357d11-f01a-47a0-9939-b0ed8e394853 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2f8eb7aec3c436fbfb1a0eb8d3df0b6)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:26.003530 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39357d11-f01a-47a0-9939-b0ed8e394853 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:26.061387 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:26.070035 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 986/1974] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:39:25 2022] POST /identity/v3/projects => generated 387 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:39:26.073886 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc20795-d344-42e3-8648-e95556437793 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:26.121692 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-39357d11-f01a-47a0-9939-b0ed8e394853 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2f8eb7aec3c436fbfb1a0eb8d3df0b6', 'request_id': 'req-39357d11-f01a-47a0-9939-b0ed8e394853'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:26.136604 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-39357d11-f01a-47a0-9939-b0ed8e394853 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2f8eb7aec3c436fbfb1a0eb8d3df0b6', 'request_id': 'req-39357d11-f01a-47a0-9939-b0ed8e394853'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:26.167311 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc20795-d344-42e3-8648-e95556437793 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:26.186889 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-39357d11-f01a-47a0-9939-b0ed8e394853 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2f8eb7aec3c436fbfb1a0eb8d3df0b6', 'request_id': 'req-39357d11-f01a-47a0-9939-b0ed8e394853'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:26.208670 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-39357d11-f01a-47a0-9939-b0ed8e394853 admin admin] The token cache is being invalidate because project a2f8eb7aec3c436fbfb1a0eb8d3df0b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:39:26.209802 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-39357d11-f01a-47a0-9939-b0ed8e394853 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:26.213158 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-39357d11-f01a-47a0-9939-b0ed8e394853 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:26.215781 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 989/1975] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:39:25 2022] DELETE /identity/v3/projects/a2f8eb7aec3c436fbfb1a0eb8d3df0b6 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:39:26.221229 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf195bdd-bc40-49cb-9057-0c9e6460c385 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:26.221624 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf195bdd-bc40-49cb-9057-0c9e6460c385 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:26.221976 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf195bdd-bc40-49cb-9057-0c9e6460c385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:26.239485 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c068e8c2-d30a-4aaa-911f-cc75e14bdb3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:26.240502 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c068e8c2-d30a-4aaa-911f-cc75e14bdb3d admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:26.241789 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c068e8c2-d30a-4aaa-911f-cc75e14bdb3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:26.243844 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c068e8c2-d30a-4aaa-911f-cc75e14bdb3d admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:26.243844 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c068e8c2-d30a-4aaa-911f-cc75e14bdb3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:26.246002 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c068e8c2-d30a-4aaa-911f-cc75e14bdb3d admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:26.249344 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-cf195bdd-bc40-49cb-9057-0c9e6460c385 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:39:26.251082 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-cf195bdd-bc40-49cb-9057-0c9e6460c385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:26.263596 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 990/1976] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:39:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 07:39:26.271557 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a77cda0-0647-40d7-9183-326368a06b6d None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:26.271848 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a77cda0-0647-40d7-9183-326368a06b6d None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:26.272124 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a77cda0-0647-40d7-9183-326368a06b6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:26.295075 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-0a77cda0-0647-40d7-9183-326368a06b6d None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:39:26.296302 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-0a77cda0-0647-40d7-9183-326368a06b6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:26.304894 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 991/1977] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:39:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 07:39:26.309806 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a77cda0-0647-40d7-9183-326368a06b6d None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:26.319594 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a77cda0-0647-40d7-9183-326368a06b6d None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:26.324546 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d073277f-a17c-4bab-b50d-cc79ca1d96e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:26.325157 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d073277f-a17c-4bab-b50d-cc79ca1d96e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:26.325438 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d073277f-a17c-4bab-b50d-cc79ca1d96e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:26.325742 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d073277f-a17c-4bab-b50d-cc79ca1d96e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:26.326388 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d073277f-a17c-4bab-b50d-cc79ca1d96e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:39:26.326697 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d073277f-a17c-4bab-b50d-cc79ca1d96e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:26.328366 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d073277f-a17c-4bab-b50d-cc79ca1d96e2 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:26.340952 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 992/1978] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 15 07:39:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:39:26.345510 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d073277f-a17c-4bab-b50d-cc79ca1d96e2 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:26.350219 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 987/1979] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:39:26 2022] POST /identity/v3/users => generated 300 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 07:39:26.354966 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c068e8c2-d30a-4aaa-911f-cc75e14bdb3d admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:26.357310 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-c068e8c2-d30a-4aaa-911f-cc75e14bdb3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:26.359227 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d073277f-a17c-4bab-b50d-cc79ca1d96e2 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:26.365156 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068205ac-6c3c-4ca6-90df-7b7924e16782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:26.365156 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068205ac-6c3c-4ca6-90df-7b7924e16782 admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:26.365406 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068205ac-6c3c-4ca6-90df-7b7924e16782 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:26.365772 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068205ac-6c3c-4ca6-90df-7b7924e16782 admin admin] PATH_INFO: `/v3/projects` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:26.366641 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068205ac-6c3c-4ca6-90df-7b7924e16782 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:26.373247 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068205ac-6c3c-4ca6-90df-7b7924e16782 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:26.375516 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c068e8c2-d30a-4aaa-911f-cc75e14bdb3d admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:26.382662 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89094ecb-7c5f-46fe-8455-dac646a94439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:26.383816 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89094ecb-7c5f-46fe-8455-dac646a94439 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:26.384662 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89094ecb-7c5f-46fe-8455-dac646a94439 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:26.384662 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89094ecb-7c5f-46fe-8455-dac646a94439 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:26.385555 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89094ecb-7c5f-46fe-8455-dac646a94439 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:26.387658 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89094ecb-7c5f-46fe-8455-dac646a94439 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:26.400278 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 988/1980] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:39:26 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:39:26.407814 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89094ecb-7c5f-46fe-8455-dac646a94439 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:26.410242 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:26.425043 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89094ecb-7c5f-46fe-8455-dac646a94439 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:26.429822 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058ab9d9-e639-42de-ad7c-2e02884ea053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:26.431614 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058ab9d9-e639-42de-ad7c-2e02884ea053 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:26.432055 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058ab9d9-e639-42de-ad7c-2e02884ea053 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:26.432318 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058ab9d9-e639-42de-ad7c-2e02884ea053 admin admin] PATH_INFO: `/v3/projects/42fb97e9fb2f4b12a563d161a6bec6cc/users/878d1ce4c716423ba2d33e35c30c178e/roles/6bbb643e0d8d4bc0a9b27c26edf59216` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:26.456213 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058ab9d9-e639-42de-ad7c-2e02884ea053 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42fb97e9fb2f4b12a563d161a6bec6cc, user_id=878d1ce4c716423ba2d33e35c30c178e, role_id=6bbb643e0d8d4bc0a9b27c26edf59216)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:26.461977 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058ab9d9-e639-42de-ad7c-2e02884ea053 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:26.521986 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 993/1981] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:39:26 2022] POST /identity/v3/projects => generated 359 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:39:26.523528 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068205ac-6c3c-4ca6-90df-7b7924e16782 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:26.575471 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 989/1982] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:39:26 2022] PUT /identity/v3/projects/42fb97e9fb2f4b12a563d161a6bec6cc/users/878d1ce4c716423ba2d33e35c30c178e/roles/6bbb643e0d8d4bc0a9b27c26edf59216 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 07:39:26.580231 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058ab9d9-e639-42de-ad7c-2e02884ea053 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:26.626995 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068205ac-6c3c-4ca6-90df-7b7924e16782 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:26.665099 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058ab9d9-e639-42de-ad7c-2e02884ea053 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:26.665722 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:26.676469 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f8d81e-7062-4996-a114-84ea34f40aa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:26.677462 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f8d81e-7062-4996-a114-84ea34f40aa4 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:26.678132 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f8d81e-7062-4996-a114-84ea34f40aa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:26.678664 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f8d81e-7062-4996-a114-84ea34f40aa4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:26.679523 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f8d81e-7062-4996-a114-84ea34f40aa4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:26.682146 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f8d81e-7062-4996-a114-84ea34f40aa4 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:26.697763 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 990/1983] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:39:26 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 07:39:26.702004 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f8d81e-7062-4996-a114-84ea34f40aa4 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:26.712444 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e73225-7536-49ea-8b68-24fe290c912a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:26.713310 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e73225-7536-49ea-8b68-24fe290c912a admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:26.713620 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e73225-7536-49ea-8b68-24fe290c912a admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:26.713906 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e73225-7536-49ea-8b68-24fe290c912a admin admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:26.714607 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e73225-7536-49ea-8b68-24fe290c912a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:26.716781 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e73225-7536-49ea-8b68-24fe290c912a admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:26.717115 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f8d81e-7062-4996-a114-84ea34f40aa4 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:26.722698 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962d182f-03d0-44d1-ad1f-b41b3fe38b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:26.723637 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962d182f-03d0-44d1-ad1f-b41b3fe38b95 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:26.724258 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962d182f-03d0-44d1-ad1f-b41b3fe38b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:26.724762 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962d182f-03d0-44d1-ad1f-b41b3fe38b95 admin admin] PATH_INFO: `/v3/projects/42fb97e9fb2f4b12a563d161a6bec6cc/users/878d1ce4c716423ba2d33e35c30c178e/roles/6bbb643e0d8d4bc0a9b27c26edf59216` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:26.729147 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962d182f-03d0-44d1-ad1f-b41b3fe38b95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42fb97e9fb2f4b12a563d161a6bec6cc, user_id=878d1ce4c716423ba2d33e35c30c178e, role_id=6bbb643e0d8d4bc0a9b27c26edf59216)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:26.732115 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962d182f-03d0-44d1-ad1f-b41b3fe38b95 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:26.746745 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 991/1984] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:39:26 2022] PUT /identity/v3/projects/42fb97e9fb2f4b12a563d161a6bec6cc/users/878d1ce4c716423ba2d33e35c30c178e/roles/6bbb643e0d8d4bc0a9b27c26edf59216 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 07:39:26.762082 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71db5ca-dc48-4179-849c-c8b566e42e09 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:26.762596 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71db5ca-dc48-4179-849c-c8b566e42e09 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:26.763157 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71db5ca-dc48-4179-849c-c8b566e42e09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:26.806767 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 994/1985] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:39:26 2022] POST /identity/v3/users => generated 280 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 07:39:26.810745 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e73225-7536-49ea-8b68-24fe290c912a admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:26.817532 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-b71db5ca-dc48-4179-849c-c8b566e42e09 None None] MFA Rules not processed for user `878d1ce4c716423ba2d33e35c30c178e`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:39:26.818822 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-b71db5ca-dc48-4179-849c-c8b566e42e09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:26.828963 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:26.829175 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e73225-7536-49ea-8b68-24fe290c912a admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:26.885944 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:26.913039 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:26.940862 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1936a554-80f4-443d-b91c-79662eb15c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:26.941810 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1936a554-80f4-443d-b91c-79662eb15c75 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:26.946335 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 992/1986] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:39:26 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:39:26.947371 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1936a554-80f4-443d-b91c-79662eb15c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:26.947881 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1936a554-80f4-443d-b91c-79662eb15c75 admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:26.949523 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1936a554-80f4-443d-b91c-79662eb15c75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:26.956244 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71db5ca-dc48-4179-849c-c8b566e42e09 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:26.959757 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-b71db5ca-dc48-4179-849c-c8b566e42e09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:26.961740 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1936a554-80f4-443d-b91c-79662eb15c75 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:26.984940 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 995/1987] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:39:26 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:39:26.985965 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1936a554-80f4-443d-b91c-79662eb15c75 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:27.000073 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1936a554-80f4-443d-b91c-79662eb15c75 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:27.005645 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2e4868-547c-4d89-848e-5fe2f76dda84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:27.006527 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2e4868-547c-4d89-848e-5fe2f76dda84 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:27.006916 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2e4868-547c-4d89-848e-5fe2f76dda84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:27.007288 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2e4868-547c-4d89-848e-5fe2f76dda84 admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:27.009140 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2e4868-547c-4d89-848e-5fe2f76dda84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:27.011357 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2e4868-547c-4d89-848e-5fe2f76dda84 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:27.043095 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 996/1988] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:39:26 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:39:27.050552 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2e4868-547c-4d89-848e-5fe2f76dda84 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:27.063245 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:27.074199 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71db5ca-dc48-4179-849c-c8b566e42e09 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:27.074903 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2e4868-547c-4d89-848e-5fe2f76dda84 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:27.080992 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f31faed-2180-4c80-aced-98789a6d494d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:27.081871 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f31faed-2180-4c80-aced-98789a6d494d admin admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:27.082252 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f31faed-2180-4c80-aced-98789a6d494d admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:27.092591 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f31faed-2180-4c80-aced-98789a6d494d admin admin] PATH_INFO: `/v3/projects/092450aeb4f94a5e8224bac10468b516/users/27b380f3c2314a118238d76ddd519360/roles/053e12af84e943e699460a9d76c2072b` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:27.092747 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baf0c64-f659-4a06-b12c-1da0b8b1a770 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecf6642b7b84d3c9756bb41ab3349d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:27.092747 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baf0c64-f659-4a06-b12c-1da0b8b1a770 service cinder] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:27.092747 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baf0c64-f659-4a06-b12c-1da0b8b1a770 service cinder] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:27.094733 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baf0c64-f659-4a06-b12c-1da0b8b1a770 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:27.105880 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f31faed-2180-4c80-aced-98789a6d494d admin admin] RBAC: Authorizing `identity:create_grant(project_id=092450aeb4f94a5e8224bac10468b516, user_id=27b380f3c2314a118238d76ddd519360, role_id=053e12af84e943e699460a9d76c2072b)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:27.112523 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f31faed-2180-4c80-aced-98789a6d494d admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:27.116249 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baf0c64-f659-4a06-b12c-1da0b8b1a770 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:27.119015 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baf0c64-f659-4a06-b12c-1da0b8b1a770 service cinder] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:27.150071 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 993/1989] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:39:26 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:39:27.196427 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 997/1990] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:39:27 2022] PUT /identity/v3/projects/092450aeb4f94a5e8224bac10468b516/users/27b380f3c2314a118238d76ddd519360/roles/053e12af84e943e699460a9d76c2072b => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:39:27.207507 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbbc9a1-5f61-4125-92ca-d0ffc70d0ff3 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:27.207849 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbbc9a1-5f61-4125-92ca-d0ffc70d0ff3 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:27.208221 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbbc9a1-5f61-4125-92ca-d0ffc70d0ff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:27.259949 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-1fbbc9a1-5f61-4125-92ca-d0ffc70d0ff3 None None] MFA Rules not processed for user `27b380f3c2314a118238d76ddd519360`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:39:27.261294 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-1fbbc9a1-5f61-4125-92ca-d0ffc70d0ff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:27.269003 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:27.338965 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 994/1991] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:39:27 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:39:27.523660 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:27.650539 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f31faed-2180-4c80-aced-98789a6d494d admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:27.711269 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f31faed-2180-4c80-aced-98789a6d494d admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:27.715780 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592fb819-3257-470b-9826-d5e35527150f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecf6642b7b84d3c9756bb41ab3349d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:27.716803 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592fb819-3257-470b-9826-d5e35527150f service cinder] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:27.717363 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592fb819-3257-470b-9826-d5e35527150f service cinder] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:27.717875 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592fb819-3257-470b-9826-d5e35527150f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:27.730310 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592fb819-3257-470b-9826-d5e35527150f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:27.732821 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592fb819-3257-470b-9826-d5e35527150f service cinder] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:27.744007 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 998/1992] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:39:27 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:39:27.751953 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:27.760799 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbbc9a1-5f61-4125-92ca-d0ffc70d0ff3 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:27.836343 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbbc9a1-5f61-4125-92ca-d0ffc70d0ff3 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:27.842544 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b502013a-5798-4811-9dca-c65c4c7d8cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:27.843203 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b502013a-5798-4811-9dca-c65c4c7d8cc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:27.848022 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b502013a-5798-4811-9dca-c65c4c7d8cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:27.848022 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b502013a-5798-4811-9dca-c65c4c7d8cc8 admin admin] PATH_INFO: `/v3/users/27b380f3c2314a118238d76ddd519360` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:27.848022 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b502013a-5798-4811-9dca-c65c4c7d8cc8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27b380f3c2314a118238d76ddd519360)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:27.848022 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b502013a-5798-4811-9dca-c65c4c7d8cc8 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:27.895591 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:27.961194 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:28.011049 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-b502013a-5798-4811-9dca-c65c4c7d8cc8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27b380f3c2314a118238d76ddd519360'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:28.011675 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-b502013a-5798-4811-9dca-c65c4c7d8cc8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27b380f3c2314a118238d76ddd519360'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:28.064021 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-b502013a-5798-4811-9dca-c65c4c7d8cc8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27b380f3c2314a118238d76ddd519360'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:28.064668 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-b502013a-5798-4811-9dca-c65c4c7d8cc8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27b380f3c2314a118238d76ddd519360'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:28.074361 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-b502013a-5798-4811-9dca-c65c4c7d8cc8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27b380f3c2314a118238d76ddd519360'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:28.093204 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 995/1993] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:39:27 2022] DELETE /identity/v3/users/27b380f3c2314a118238d76ddd519360 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:39:28.097961 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f537fc9-8406-4979-9e12-1086ee2d1126 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:28.098245 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f537fc9-8406-4979-9e12-1086ee2d1126 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:28.098560 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f537fc9-8406-4979-9e12-1086ee2d1126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:28.119822 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-2f537fc9-8406-4979-9e12-1086ee2d1126 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:39:28.121030 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-2f537fc9-8406-4979-9e12-1086ee2d1126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:28.181472 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:28.184182 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 999/1994] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:39:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:39:28.194306 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b502013a-5798-4811-9dca-c65c4c7d8cc8 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:28.197087 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-b502013a-5798-4811-9dca-c65c4c7d8cc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:28.227424 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:28.256506 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b502013a-5798-4811-9dca-c65c4c7d8cc8 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:28.261036 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66fdf0b-c886-4f36-9a71-b856bff94ee1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:28.261745 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66fdf0b-c886-4f36-9a71-b856bff94ee1 service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:28.261977 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66fdf0b-c886-4f36-9a71-b856bff94ee1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:28.262256 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66fdf0b-c886-4f36-9a71-b856bff94ee1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:28.271302 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66fdf0b-c886-4f36-9a71-b856bff94ee1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:28.272720 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66fdf0b-c886-4f36-9a71-b856bff94ee1 service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:28.282577 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 996/1995] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:39:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:39:28.357982 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:28.409711 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:28.440585 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f537fc9-8406-4979-9e12-1086ee2d1126 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:28.440585 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-2f537fc9-8406-4979-9e12-1086ee2d1126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:28.505708 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f537fc9-8406-4979-9e12-1086ee2d1126 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:28.510587 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580f5c3d-c968-4b07-a066-e7125a68ed07 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecf6642b7b84d3c9756bb41ab3349d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:28.517751 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580f5c3d-c968-4b07-a066-e7125a68ed07 service cinder] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:28.517751 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580f5c3d-c968-4b07-a066-e7125a68ed07 service cinder] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:28.517751 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580f5c3d-c968-4b07-a066-e7125a68ed07 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:28.517751 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-580f5c3d-c968-4b07-a066-e7125a68ed07 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:28.578265 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580f5c3d-c968-4b07-a066-e7125a68ed07 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:28.579736 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580f5c3d-c968-4b07-a066-e7125a68ed07 service cinder] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:28.590181 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1000/1996] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:39:28 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:39:28.686114 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66fdf0b-c886-4f36-9a71-b856bff94ee1 service neutron] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:28.688520 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-b66fdf0b-c886-4f36-9a71-b856bff94ee1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:28.702839 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66fdf0b-c886-4f36-9a71-b856bff94ee1 service neutron] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:28.709063 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0d0278-9261-4aec-8588-34bbdedaa456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:28.709948 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0d0278-9261-4aec-8588-34bbdedaa456 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:28.710235 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0d0278-9261-4aec-8588-34bbdedaa456 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:28.710513 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0d0278-9261-4aec-8588-34bbdedaa456 admin admin] PATH_INFO: `/v3/projects/092450aeb4f94a5e8224bac10468b516` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:28.712558 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0d0278-9261-4aec-8588-34bbdedaa456 admin admin] RBAC: Authorizing `identity:delete_project(project_id=092450aeb4f94a5e8224bac10468b516)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:28.714665 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0d0278-9261-4aec-8588-34bbdedaa456 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:28.870257 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-af0d0278-9261-4aec-8588-34bbdedaa456 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '092450aeb4f94a5e8224bac10468b516', 'request_id': 'req-af0d0278-9261-4aec-8588-34bbdedaa456'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:28.877281 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-af0d0278-9261-4aec-8588-34bbdedaa456 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '092450aeb4f94a5e8224bac10468b516', 'request_id': 'req-af0d0278-9261-4aec-8588-34bbdedaa456'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:28.972679 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-af0d0278-9261-4aec-8588-34bbdedaa456 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '092450aeb4f94a5e8224bac10468b516', 'request_id': 'req-af0d0278-9261-4aec-8588-34bbdedaa456'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:28.973232 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:28.984759 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-af0d0278-9261-4aec-8588-34bbdedaa456 admin admin] The token cache is being invalidate because project 092450aeb4f94a5e8224bac10468b516 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:39:28.985063 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-af0d0278-9261-4aec-8588-34bbdedaa456 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:28.985747 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-af0d0278-9261-4aec-8588-34bbdedaa456 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:28.987242 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 997/1997] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:39:28 2022] DELETE /identity/v3/projects/092450aeb4f94a5e8224bac10468b516 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:39:28.994349 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcf3497-ffd5-4ee9-ac61-f1c81611ff0b None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:28.994661 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcf3497-ffd5-4ee9-ac61-f1c81611ff0b None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:28.994931 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcf3497-ffd5-4ee9-ac61-f1c81611ff0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:29.014354 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-2fcf3497-ffd5-4ee9-ac61-f1c81611ff0b None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:39:29.015529 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-2fcf3497-ffd5-4ee9-ac61-f1c81611ff0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:29.077614 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1001/1998] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:39:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:39:29.081603 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0d2a14-f49c-45c0-ae16-1a4650743633 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:29.081915 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0d2a14-f49c-45c0-ae16-1a4650743633 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:29.082197 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0d2a14-f49c-45c0-ae16-1a4650743633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:29.111469 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-ed0d2a14-f49c-45c0-ae16-1a4650743633 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:39:29.112697 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-ed0d2a14-f49c-45c0-ae16-1a4650743633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:29.122226 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 998/1999] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:39:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:39:29.126557 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcf3497-ffd5-4ee9-ac61-f1c81611ff0b None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:29.137214 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcf3497-ffd5-4ee9-ac61-f1c81611ff0b None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:29.142302 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedf36b1-33b2-49c7-94f8-894ff63f54af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:29.143006 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedf36b1-33b2-49c7-94f8-894ff63f54af admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:29.143287 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedf36b1-33b2-49c7-94f8-894ff63f54af admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:29.143621 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedf36b1-33b2-49c7-94f8-894ff63f54af admin admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:29.144422 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedf36b1-33b2-49c7-94f8-894ff63f54af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:39:29.144754 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedf36b1-33b2-49c7-94f8-894ff63f54af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:29.150168 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedf36b1-33b2-49c7-94f8-894ff63f54af admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:29.156183 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0d2a14-f49c-45c0-ae16-1a4650743633 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:29.157593 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-ed0d2a14-f49c-45c0-ae16-1a4650743633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:29.175570 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1002/2000] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 15 07:39:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:39:29.181738 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedf36b1-33b2-49c7-94f8-894ff63f54af admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:29.192099 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedf36b1-33b2-49c7-94f8-894ff63f54af admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:29.198116 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd226db-7b7a-476d-a4fd-7a23519564ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:29.198807 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd226db-7b7a-476d-a4fd-7a23519564ea admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:29.199285 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd226db-7b7a-476d-a4fd-7a23519564ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:29.199636 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd226db-7b7a-476d-a4fd-7a23519564ea admin admin] PATH_INFO: `/v3/projects` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:29.200338 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd226db-7b7a-476d-a4fd-7a23519564ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:29.202348 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd226db-7b7a-476d-a4fd-7a23519564ea admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:29.211263 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:29.228842 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0d2a14-f49c-45c0-ae16-1a4650743633 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:29.232666 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece625b2-23c0-4d85-b5fc-95fa20fa908f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecf6642b7b84d3c9756bb41ab3349d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:29.233314 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece625b2-23c0-4d85-b5fc-95fa20fa908f service cinder] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:29.233645 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece625b2-23c0-4d85-b5fc-95fa20fa908f service cinder] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:29.233919 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece625b2-23c0-4d85-b5fc-95fa20fa908f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:29.235696 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-ece625b2-23c0-4d85-b5fc-95fa20fa908f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:29.291077 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:29.294084 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1003/2001] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:39:29 2022] POST /identity/v3/projects => generated 375 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 07:39:29.298412 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd226db-7b7a-476d-a4fd-7a23519564ea admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:29.301427 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece625b2-23c0-4d85-b5fc-95fa20fa908f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:29.303020 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece625b2-23c0-4d85-b5fc-95fa20fa908f service cinder] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:29.313336 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 999/2002] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:39:29 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 07:39:29.376418 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd226db-7b7a-476d-a4fd-7a23519564ea admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:29.381301 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91e66a2-f0c5-4d12-aeb9-c6e5057be1d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:29.382090 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91e66a2-f0c5-4d12-aeb9-c6e5057be1d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:29.382433 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91e66a2-f0c5-4d12-aeb9-c6e5057be1d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:29.382714 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91e66a2-f0c5-4d12-aeb9-c6e5057be1d9 admin admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:29.383900 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91e66a2-f0c5-4d12-aeb9-c6e5057be1d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:29.386125 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91e66a2-f0c5-4d12-aeb9-c6e5057be1d9 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:29.546452 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1004/2003] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:39:29 2022] POST /identity/v3/users => generated 288 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:39:29.550613 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece625b2-23c0-4d85-b5fc-95fa20fa908f service cinder] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:29.561626 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece625b2-23c0-4d85-b5fc-95fa20fa908f service cinder] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:29.566296 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77ec528-d893-4d93-b7cb-cfefc365c5a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:29.566902 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77ec528-d893-4d93-b7cb-cfefc365c5a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:29.567250 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77ec528-d893-4d93-b7cb-cfefc365c5a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:29.567532 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77ec528-d893-4d93-b7cb-cfefc365c5a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:29.568218 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77ec528-d893-4d93-b7cb-cfefc365c5a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:29.569845 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77ec528-d893-4d93-b7cb-cfefc365c5a6 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:29.580281 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1000/2004] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:39:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:39:29.584056 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91e66a2-f0c5-4d12-aeb9-c6e5057be1d9 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:29.596111 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91e66a2-f0c5-4d12-aeb9-c6e5057be1d9 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:29.600707 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0100a8-cf2a-4306-9436-2736f1f8e419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:29.601311 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0100a8-cf2a-4306-9436-2736f1f8e419 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:29.601619 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0100a8-cf2a-4306-9436-2736f1f8e419 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:29.602063 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0100a8-cf2a-4306-9436-2736f1f8e419 admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:29.602783 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0100a8-cf2a-4306-9436-2736f1f8e419 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:29.604361 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0100a8-cf2a-4306-9436-2736f1f8e419 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:29.614339 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1005/2005] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:39:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:39:29.622936 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77ec528-d893-4d93-b7cb-cfefc365c5a6 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:29.633957 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77ec528-d893-4d93-b7cb-cfefc365c5a6 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:29.638641 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:29.640482 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244da6d5-cf63-47d6-a31b-4de807cde91b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:29.641144 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244da6d5-cf63-47d6-a31b-4de807cde91b admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:29.641448 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244da6d5-cf63-47d6-a31b-4de807cde91b admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:29.641850 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244da6d5-cf63-47d6-a31b-4de807cde91b admin admin] PATH_INFO: `/v3/projects/7b9807a3e6dd42a385830ad8d7f735dc/users/f57fba98553746e5842b037309f2284e/roles/053e12af84e943e699460a9d76c2072b` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:29.657040 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244da6d5-cf63-47d6-a31b-4de807cde91b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b9807a3e6dd42a385830ad8d7f735dc, user_id=f57fba98553746e5842b037309f2284e, role_id=053e12af84e943e699460a9d76c2072b)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:29.659832 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244da6d5-cf63-47d6-a31b-4de807cde91b admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:29.671964 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:29.738500 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1001/2006] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:39:29 2022] PUT /identity/v3/projects/7b9807a3e6dd42a385830ad8d7f735dc/users/f57fba98553746e5842b037309f2284e/roles/053e12af84e943e699460a9d76c2072b => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:39:29.749590 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a737948b-0965-4b8d-9b58-1b7c60d806b8 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:29.749893 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a737948b-0965-4b8d-9b58-1b7c60d806b8 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:29.750173 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a737948b-0965-4b8d-9b58-1b7c60d806b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:29.799350 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-a737948b-0965-4b8d-9b58-1b7c60d806b8 None None] MFA Rules not processed for user `f57fba98553746e5842b037309f2284e`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:39:29.800572 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-a737948b-0965-4b8d-9b58-1b7c60d806b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:29.875500 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1006/2007] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:39:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:39:29.880113 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244da6d5-cf63-47d6-a31b-4de807cde91b admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:29.950720 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244da6d5-cf63-47d6-a31b-4de807cde91b admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:29.959512 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3b0948-280b-4752-954d-e6fa17fbed27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:29.960429 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3b0948-280b-4752-954d-e6fa17fbed27 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:29.960810 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3b0948-280b-4752-954d-e6fa17fbed27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:29.961097 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3b0948-280b-4752-954d-e6fa17fbed27 admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:29.962048 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3b0948-280b-4752-954d-e6fa17fbed27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:29.964035 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3b0948-280b-4752-954d-e6fa17fbed27 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:30.052264 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a737948b-0965-4b8d-9b58-1b7c60d806b8 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:30.059762 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1002/2008] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:39:29 2022] POST /identity/v3/projects => generated 373 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:39:30.072147 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a737948b-0965-4b8d-9b58-1b7c60d806b8 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:30.148470 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeb34a9-2bbb-459b-8ec2-2215c68c4054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:30.149754 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeb34a9-2bbb-459b-8ec2-2215c68c4054 admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:30.150053 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeb34a9-2bbb-459b-8ec2-2215c68c4054 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:30.150331 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeb34a9-2bbb-459b-8ec2-2215c68c4054 admin admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:30.150971 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeb34a9-2bbb-459b-8ec2-2215c68c4054 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:30.153077 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeb34a9-2bbb-459b-8ec2-2215c68c4054 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:30.161688 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:30.243545 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:30.249929 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1007/2009] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:39:30 2022] POST /identity/v3/users => generated 293 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:39:30.254134 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3b0948-280b-4752-954d-e6fa17fbed27 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:30.264524 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3b0948-280b-4752-954d-e6fa17fbed27 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:30.269153 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b458ec6d-d957-495b-9ed3-ccb30256b4ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:30.269782 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b458ec6d-d957-495b-9ed3-ccb30256b4ad admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:30.270068 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b458ec6d-d957-495b-9ed3-ccb30256b4ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:30.270344 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b458ec6d-d957-495b-9ed3-ccb30256b4ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:30.270986 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b458ec6d-d957-495b-9ed3-ccb30256b4ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:30.272587 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b458ec6d-d957-495b-9ed3-ccb30256b4ad admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:30.285223 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1003/2010] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:39:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:39:30.290205 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeb34a9-2bbb-459b-8ec2-2215c68c4054 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:30.301861 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeb34a9-2bbb-459b-8ec2-2215c68c4054 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:30.307399 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf014d01-4591-4a9c-a73f-1e0b2b84ef29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:30.308259 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf014d01-4591-4a9c-a73f-1e0b2b84ef29 admin admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:30.308573 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf014d01-4591-4a9c-a73f-1e0b2b84ef29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:30.308866 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf014d01-4591-4a9c-a73f-1e0b2b84ef29 admin admin] PATH_INFO: `/v3/projects/9e1196883ef94f2195af5aa29c1ec1fa/users/e90a5a0fc4a240549e9a19ce1a97efe1/roles/6bbb643e0d8d4bc0a9b27c26edf59216` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:30.325568 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf014d01-4591-4a9c-a73f-1e0b2b84ef29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e1196883ef94f2195af5aa29c1ec1fa, user_id=e90a5a0fc4a240549e9a19ce1a97efe1, role_id=6bbb643e0d8d4bc0a9b27c26edf59216)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:30.328255 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf014d01-4591-4a9c-a73f-1e0b2b84ef29 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:30.444472 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1008/2011] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:39:30 2022] PUT /identity/v3/projects/9e1196883ef94f2195af5aa29c1ec1fa/users/e90a5a0fc4a240549e9a19ce1a97efe1/roles/6bbb643e0d8d4bc0a9b27c26edf59216 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:39:30.448640 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b458ec6d-d957-495b-9ed3-ccb30256b4ad admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:30.459593 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b458ec6d-d957-495b-9ed3-ccb30256b4ad admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:30.530386 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e2f188-d584-42bd-a6fa-efb36d715333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:30.531062 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e2f188-d584-42bd-a6fa-efb36d715333 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:30.531645 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e2f188-d584-42bd-a6fa-efb36d715333 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:30.532272 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e2f188-d584-42bd-a6fa-efb36d715333 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:30.533200 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e2f188-d584-42bd-a6fa-efb36d715333 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:30.534907 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e2f188-d584-42bd-a6fa-efb36d715333 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:30.548706 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1004/2012] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:39:30 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:39:30.553542 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf014d01-4591-4a9c-a73f-1e0b2b84ef29 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:30.556307 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:30.566038 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf014d01-4591-4a9c-a73f-1e0b2b84ef29 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:30.573567 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b44824-31f1-44dc-bd29-f1194d5d805f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:30.574270 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b44824-31f1-44dc-bd29-f1194d5d805f admin admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:30.574550 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b44824-31f1-44dc-bd29-f1194d5d805f admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:30.574822 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b44824-31f1-44dc-bd29-f1194d5d805f admin admin] PATH_INFO: `/v3/projects/9e1196883ef94f2195af5aa29c1ec1fa/users/e90a5a0fc4a240549e9a19ce1a97efe1/roles/6bbb643e0d8d4bc0a9b27c26edf59216` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:30.577025 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b44824-31f1-44dc-bd29-f1194d5d805f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e1196883ef94f2195af5aa29c1ec1fa, user_id=e90a5a0fc4a240549e9a19ce1a97efe1, role_id=6bbb643e0d8d4bc0a9b27c26edf59216)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:30.579739 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b44824-31f1-44dc-bd29-f1194d5d805f admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:30.593051 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1009/2013] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:39:30 2022] PUT /identity/v3/projects/9e1196883ef94f2195af5aa29c1ec1fa/users/e90a5a0fc4a240549e9a19ce1a97efe1/roles/6bbb643e0d8d4bc0a9b27c26edf59216 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:39:30.605401 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e435a57f-ef96-4f7f-b8ee-c1963c742e57 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:30.605733 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e435a57f-ef96-4f7f-b8ee-c1963c742e57 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:30.605990 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e435a57f-ef96-4f7f-b8ee-c1963c742e57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:30.647514 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-e435a57f-ef96-4f7f-b8ee-c1963c742e57 None None] MFA Rules not processed for user `e90a5a0fc4a240549e9a19ce1a97efe1`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:39:30.648730 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-e435a57f-ef96-4f7f-b8ee-c1963c742e57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:30.740582 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1005/2014] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:39:30 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:39:30.754418 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b44824-31f1-44dc-bd29-f1194d5d805f admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:30.818288 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b44824-31f1-44dc-bd29-f1194d5d805f admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:30.822805 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b56e892-1e77-4dd5-846b-d7265242d5af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecf6642b7b84d3c9756bb41ab3349d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:30.823424 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b56e892-1e77-4dd5-846b-d7265242d5af service cinder] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:30.823699 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b56e892-1e77-4dd5-846b-d7265242d5af service cinder] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:30.823971 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b56e892-1e77-4dd5-846b-d7265242d5af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:30.890087 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b56e892-1e77-4dd5-846b-d7265242d5af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:30.891451 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b56e892-1e77-4dd5-846b-d7265242d5af service cinder] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:30.901257 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1010/2015] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:39:30 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:39:30.995890 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:31.064253 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:31.278606 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:31.312441 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:31.884411 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:31.986756 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:32.133086 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:32.155805 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:32.178682 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:32.287133 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:36.558029 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e435a57f-ef96-4f7f-b8ee-c1963c742e57 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:36.631654 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e435a57f-ef96-4f7f-b8ee-c1963c742e57 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:36.637734 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f235c8-bce9-403e-b101-0be29e14d3e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:36.638392 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f235c8-bce9-403e-b101-0be29e14d3e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:36.638680 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f235c8-bce9-403e-b101-0be29e14d3e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:36.639382 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f235c8-bce9-403e-b101-0be29e14d3e3 admin admin] PATH_INFO: `/v3/users/f57fba98553746e5842b037309f2284e` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:36.640550 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f235c8-bce9-403e-b101-0be29e14d3e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f57fba98553746e5842b037309f2284e)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:36.642521 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f235c8-bce9-403e-b101-0be29e14d3e3 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:36.894306 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-f9f235c8-bce9-403e-b101-0be29e14d3e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f57fba98553746e5842b037309f2284e'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:36.894957 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-f9f235c8-bce9-403e-b101-0be29e14d3e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f57fba98553746e5842b037309f2284e'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:36.980816 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-f9f235c8-bce9-403e-b101-0be29e14d3e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f57fba98553746e5842b037309f2284e'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:36.981474 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-f9f235c8-bce9-403e-b101-0be29e14d3e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f57fba98553746e5842b037309f2284e'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:36.989379 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-f9f235c8-bce9-403e-b101-0be29e14d3e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f57fba98553746e5842b037309f2284e'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:37.009021 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1006/2016] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:39:36 2022] DELETE /identity/v3/users/f57fba98553746e5842b037309f2284e => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:39:37.020471 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ae9204-65d6-4384-b1e0-7b60a745618b None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:37.020985 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ae9204-65d6-4384-b1e0-7b60a745618b None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:37.021094 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ae9204-65d6-4384-b1e0-7b60a745618b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:37.040990 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-52ae9204-65d6-4384-b1e0-7b60a745618b None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:39:37.046711 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-52ae9204-65d6-4384-b1e0-7b60a745618b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:37.112822 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1011/2017] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:39:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:39:37.116166 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f235c8-bce9-403e-b101-0be29e14d3e3 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:37.116166 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-f9f235c8-bce9-403e-b101-0be29e14d3e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:37.124974 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ae9204-65d6-4384-b1e0-7b60a745618b None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:37.126409 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-52ae9204-65d6-4384-b1e0-7b60a745618b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:37.145874 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:37.179412 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f235c8-bce9-403e-b101-0be29e14d3e3 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:37.184381 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e7ff02-527f-4b4a-8eed-95a1bdb682d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecf6642b7b84d3c9756bb41ab3349d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:37.184785 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e7ff02-527f-4b4a-8eed-95a1bdb682d7 service cinder] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:37.185070 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e7ff02-527f-4b4a-8eed-95a1bdb682d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:37.185348 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e7ff02-527f-4b4a-8eed-95a1bdb682d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:37.187158 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-17e7ff02-527f-4b4a-8eed-95a1bdb682d7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:37.189385 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ae9204-65d6-4384-b1e0-7b60a745618b None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:37.193232 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6c969b-e23c-43fb-be08-1618ff749739 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:37.194022 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6c969b-e23c-43fb-be08-1618ff749739 service neutron] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:37.194162 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6c969b-e23c-43fb-be08-1618ff749739 service neutron] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:37.194396 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6c969b-e23c-43fb-be08-1618ff749739 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:37.203872 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6c969b-e23c-43fb-be08-1618ff749739 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:37.205713 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6c969b-e23c-43fb-be08-1618ff749739 service neutron] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:37.215592 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1012/2018] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:39:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 07:39:37.247705 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e7ff02-527f-4b4a-8eed-95a1bdb682d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:37.249225 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e7ff02-527f-4b4a-8eed-95a1bdb682d7 service cinder] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:37.259215 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1007/2019] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:39:37 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:39:37.631718 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6c969b-e23c-43fb-be08-1618ff749739 service neutron] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:37.633155 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-8d6c969b-e23c-43fb-be08-1618ff749739 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:37.645964 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6c969b-e23c-43fb-be08-1618ff749739 service neutron] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:37.653325 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55416d9c-afce-4b90-ab5f-9cd89f29d38b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:37.654092 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55416d9c-afce-4b90-ab5f-9cd89f29d38b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:37.659826 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55416d9c-afce-4b90-ab5f-9cd89f29d38b admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:37.659826 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55416d9c-afce-4b90-ab5f-9cd89f29d38b admin admin] PATH_INFO: `/v3/projects/7b9807a3e6dd42a385830ad8d7f735dc` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:37.659826 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55416d9c-afce-4b90-ab5f-9cd89f29d38b admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b9807a3e6dd42a385830ad8d7f735dc)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:37.659826 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55416d9c-afce-4b90-ab5f-9cd89f29d38b admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:37.804366 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-55416d9c-afce-4b90-ab5f-9cd89f29d38b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b9807a3e6dd42a385830ad8d7f735dc', 'request_id': 'req-55416d9c-afce-4b90-ab5f-9cd89f29d38b'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:37.810693 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-55416d9c-afce-4b90-ab5f-9cd89f29d38b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b9807a3e6dd42a385830ad8d7f735dc', 'request_id': 'req-55416d9c-afce-4b90-ab5f-9cd89f29d38b'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:37.878216 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-55416d9c-afce-4b90-ab5f-9cd89f29d38b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b9807a3e6dd42a385830ad8d7f735dc', 'request_id': 'req-55416d9c-afce-4b90-ab5f-9cd89f29d38b'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:37.890572 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-55416d9c-afce-4b90-ab5f-9cd89f29d38b admin admin] The token cache is being invalidate because project 7b9807a3e6dd42a385830ad8d7f735dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:39:37.890887 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-55416d9c-afce-4b90-ab5f-9cd89f29d38b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:37.891516 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-55416d9c-afce-4b90-ab5f-9cd89f29d38b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:37.893010 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1013/2020] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:39:37 2022] DELETE /identity/v3/projects/7b9807a3e6dd42a385830ad8d7f735dc => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:39:37.897246 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e7ff02-527f-4b4a-8eed-95a1bdb682d7 service cinder] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:37.908984 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e7ff02-527f-4b4a-8eed-95a1bdb682d7 service cinder] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:37.973876 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6ee68c-c826-4805-bc9a-34bed29079ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:37.976341 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6ee68c-c826-4805-bc9a-34bed29079ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:37.976341 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6ee68c-c826-4805-bc9a-34bed29079ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:37.976341 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6ee68c-c826-4805-bc9a-34bed29079ab admin admin] PATH_INFO: `/v3/users/e90a5a0fc4a240549e9a19ce1a97efe1` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:37.976341 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6ee68c-c826-4805-bc9a-34bed29079ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=e90a5a0fc4a240549e9a19ce1a97efe1)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:37.978240 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6ee68c-c826-4805-bc9a-34bed29079ab admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:38.068150 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55416d9c-afce-4b90-ab5f-9cd89f29d38b admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:38.069658 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-55416d9c-afce-4b90-ab5f-9cd89f29d38b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:38.085216 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55416d9c-afce-4b90-ab5f-9cd89f29d38b admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:38.091932 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258c9905-03e4-4901-a9f2-d63398e2505b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:38.092657 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258c9905-03e4-4901-a9f2-d63398e2505b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:38.093011 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258c9905-03e4-4901-a9f2-d63398e2505b admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:38.093299 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258c9905-03e4-4901-a9f2-d63398e2505b admin admin] PATH_INFO: `/v3/users/d9074fedf5b346da8920438b4964fae1` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:38.094487 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258c9905-03e4-4901-a9f2-d63398e2505b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9074fedf5b346da8920438b4964fae1)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:38.096528 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258c9905-03e4-4901-a9f2-d63398e2505b admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:38.118844 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:38.186915 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-4b6ee68c-c826-4805-bc9a-34bed29079ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e90a5a0fc4a240549e9a19ce1a97efe1'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:38.187621 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-4b6ee68c-c826-4805-bc9a-34bed29079ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e90a5a0fc4a240549e9a19ce1a97efe1'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:38.258463 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-4b6ee68c-c826-4805-bc9a-34bed29079ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e90a5a0fc4a240549e9a19ce1a97efe1'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:38.259097 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-4b6ee68c-c826-4805-bc9a-34bed29079ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e90a5a0fc4a240549e9a19ce1a97efe1'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:38.267210 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-4b6ee68c-c826-4805-bc9a-34bed29079ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e90a5a0fc4a240549e9a19ce1a97efe1'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:38.273232 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-258c9905-03e4-4901-a9f2-d63398e2505b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9074fedf5b346da8920438b4964fae1'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:38.281591 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:38.286605 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1008/2021] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:39:37 2022] DELETE /identity/v3/users/e90a5a0fc4a240549e9a19ce1a97efe1 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:39:38.350627 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-258c9905-03e4-4901-a9f2-d63398e2505b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9074fedf5b346da8920438b4964fae1'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:38.351071 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-258c9905-03e4-4901-a9f2-d63398e2505b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9074fedf5b346da8920438b4964fae1'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:38.351916 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-258c9905-03e4-4901-a9f2-d63398e2505b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9074fedf5b346da8920438b4964fae1'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:38.363133 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-258c9905-03e4-4901-a9f2-d63398e2505b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9074fedf5b346da8920438b4964fae1'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:38.382640 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1014/2022] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:39:38 2022] DELETE /identity/v3/users/d9074fedf5b346da8920438b4964fae1 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 07:39:38.391585 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3230bc4d-5b22-4347-996d-4c72db0cc30d None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:38.391876 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3230bc4d-5b22-4347-996d-4c72db0cc30d None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:38.392178 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3230bc4d-5b22-4347-996d-4c72db0cc30d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:38.411876 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-3230bc4d-5b22-4347-996d-4c72db0cc30d None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:39:38.413067 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-3230bc4d-5b22-4347-996d-4c72db0cc30d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:38.472621 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1009/2023] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:39:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:39:38.482421 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258c9905-03e4-4901-a9f2-d63398e2505b admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:38.483864 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-258c9905-03e4-4901-a9f2-d63398e2505b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:38.551157 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258c9905-03e4-4901-a9f2-d63398e2505b admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:38.555427 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3319e0-4224-4732-88ad-17061b8046b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:38.556047 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3319e0-4224-4732-88ad-17061b8046b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:38.556875 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3319e0-4224-4732-88ad-17061b8046b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:38.557170 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3319e0-4224-4732-88ad-17061b8046b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:38.566861 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3319e0-4224-4732-88ad-17061b8046b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:38.568733 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3319e0-4224-4732-88ad-17061b8046b6 service neutron] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:38.578295 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1015/2024] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:39:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:39:38.721855 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3230bc4d-5b22-4347-996d-4c72db0cc30d None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:38.723631 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-3230bc4d-5b22-4347-996d-4c72db0cc30d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:38.736689 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3230bc4d-5b22-4347-996d-4c72db0cc30d None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:38.747762 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9648716-fcac-435a-a276-e805485cefc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:38.748477 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9648716-fcac-435a-a276-e805485cefc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:38.749256 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9648716-fcac-435a-a276-e805485cefc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:38.749256 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9648716-fcac-435a-a276-e805485cefc6 admin admin] PATH_INFO: `/v3/projects/9e1196883ef94f2195af5aa29c1ec1fa` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:38.750718 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9648716-fcac-435a-a276-e805485cefc6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e1196883ef94f2195af5aa29c1ec1fa)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:38.752857 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9648716-fcac-435a-a276-e805485cefc6 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:38.872537 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-c9648716-fcac-435a-a276-e805485cefc6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e1196883ef94f2195af5aa29c1ec1fa', 'request_id': 'req-c9648716-fcac-435a-a276-e805485cefc6'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:38.881521 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-c9648716-fcac-435a-a276-e805485cefc6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e1196883ef94f2195af5aa29c1ec1fa', 'request_id': 'req-c9648716-fcac-435a-a276-e805485cefc6'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:38.967308 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-c9648716-fcac-435a-a276-e805485cefc6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e1196883ef94f2195af5aa29c1ec1fa', 'request_id': 'req-c9648716-fcac-435a-a276-e805485cefc6'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:38.978791 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-c9648716-fcac-435a-a276-e805485cefc6 admin admin] The token cache is being invalidate because project 9e1196883ef94f2195af5aa29c1ec1fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:39:38.982782 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-c9648716-fcac-435a-a276-e805485cefc6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:38.982782 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-c9648716-fcac-435a-a276-e805485cefc6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:38.982782 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1010/2025] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:39:38 2022] DELETE /identity/v3/projects/9e1196883ef94f2195af5aa29c1ec1fa => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:39:38.986437 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f639272a-c52c-4b04-9966-2850ee4ae949 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:38.986728 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f639272a-c52c-4b04-9966-2850ee4ae949 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:38.987006 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f639272a-c52c-4b04-9966-2850ee4ae949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:39.007343 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-f639272a-c52c-4b04-9966-2850ee4ae949 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:39:39.010206 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-f639272a-c52c-4b04-9966-2850ee4ae949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:39.071182 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1016/2026] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:39:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:39:39.079813 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7857ccb9-2a85-4275-96a3-6938d8b783cb None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:39.080864 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7857ccb9-2a85-4275-96a3-6938d8b783cb None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:39.080864 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7857ccb9-2a85-4275-96a3-6938d8b783cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:39.104739 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-7857ccb9-2a85-4275-96a3-6938d8b783cb None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:39:39.106917 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-7857ccb9-2a85-4275-96a3-6938d8b783cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:39.121608 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1011/2027] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:39:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:39:39.125945 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f639272a-c52c-4b04-9966-2850ee4ae949 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:39.136422 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f639272a-c52c-4b04-9966-2850ee4ae949 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:39.141909 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0242823-9755-457a-98f4-d6cfbfa6fef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:39.142257 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0242823-9755-457a-98f4-d6cfbfa6fef9 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:39.142545 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0242823-9755-457a-98f4-d6cfbfa6fef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:39.142822 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0242823-9755-457a-98f4-d6cfbfa6fef9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:39.143476 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0242823-9755-457a-98f4-d6cfbfa6fef9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:39:39.143777 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0242823-9755-457a-98f4-d6cfbfa6fef9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:39.145402 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0242823-9755-457a-98f4-d6cfbfa6fef9 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:39.154812 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7857ccb9-2a85-4275-96a3-6938d8b783cb None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:39.157704 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-7857ccb9-2a85-4275-96a3-6938d8b783cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:39.171892 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7857ccb9-2a85-4275-96a3-6938d8b783cb None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:39.174465 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1017/2028] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 15 07:39:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:39:39.182585 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6df0075-18b3-40f1-a86c-cdf6461c38e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:39.184107 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6df0075-18b3-40f1-a86c-cdf6461c38e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:39.184689 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6df0075-18b3-40f1-a86c-cdf6461c38e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:39.185186 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6df0075-18b3-40f1-a86c-cdf6461c38e0 admin admin] PATH_INFO: `/v3/projects/4170e0145f1b444aaff40dafb6a645d6` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:39.186671 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6df0075-18b3-40f1-a86c-cdf6461c38e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4170e0145f1b444aaff40dafb6a645d6)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:39.188222 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0242823-9755-457a-98f4-d6cfbfa6fef9 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:39.188818 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6df0075-18b3-40f1-a86c-cdf6461c38e0 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:39.200168 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0242823-9755-457a-98f4-d6cfbfa6fef9 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:39.207023 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c69fd83-912f-4548-b59f-e1d106edba56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:39.207821 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c69fd83-912f-4548-b59f-e1d106edba56 admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:39.208278 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c69fd83-912f-4548-b59f-e1d106edba56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:39.208658 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c69fd83-912f-4548-b59f-e1d106edba56 admin admin] PATH_INFO: `/v3/projects` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:39.209716 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c69fd83-912f-4548-b59f-e1d106edba56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:39.211930 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c69fd83-912f-4548-b59f-e1d106edba56 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:39.287651 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-f6df0075-18b3-40f1-a86c-cdf6461c38e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4170e0145f1b444aaff40dafb6a645d6', 'request_id': 'req-f6df0075-18b3-40f1-a86c-cdf6461c38e0'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:39.294549 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-f6df0075-18b3-40f1-a86c-cdf6461c38e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4170e0145f1b444aaff40dafb6a645d6', 'request_id': 'req-f6df0075-18b3-40f1-a86c-cdf6461c38e0'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:39.301407 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1018/2029] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:39:39 2022] POST /identity/v3/projects => generated 369 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:39:39.305629 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c69fd83-912f-4548-b59f-e1d106edba56 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:39.315698 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:39.373000 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c69fd83-912f-4548-b59f-e1d106edba56 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:39.378886 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-f6df0075-18b3-40f1-a86c-cdf6461c38e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4170e0145f1b444aaff40dafb6a645d6', 'request_id': 'req-f6df0075-18b3-40f1-a86c-cdf6461c38e0'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:39.380888 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e86249-e39d-4892-ab7b-9250adef1b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:39.381375 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e86249-e39d-4892-ab7b-9250adef1b60 admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:39.381672 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e86249-e39d-4892-ab7b-9250adef1b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:39.381955 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e86249-e39d-4892-ab7b-9250adef1b60 admin admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:39.382727 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e86249-e39d-4892-ab7b-9250adef1b60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:39.385008 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e86249-e39d-4892-ab7b-9250adef1b60 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:39.388989 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:39.394570 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-f6df0075-18b3-40f1-a86c-cdf6461c38e0 admin admin] The token cache is being invalidate because project 4170e0145f1b444aaff40dafb6a645d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:39:39.394875 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-f6df0075-18b3-40f1-a86c-cdf6461c38e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:39.395596 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-f6df0075-18b3-40f1-a86c-cdf6461c38e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:39.397255 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1012/2030] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:39:39 2022] DELETE /identity/v3/projects/4170e0145f1b444aaff40dafb6a645d6 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:39:39.401420 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6df0075-18b3-40f1-a86c-cdf6461c38e0 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:39.402963 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-f6df0075-18b3-40f1-a86c-cdf6461c38e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:39.470351 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6df0075-18b3-40f1-a86c-cdf6461c38e0 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:39.473510 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1019/2031] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:39:39 2022] POST /identity/v3/users => generated 285 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 07:39:39.474756 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e86249-e39d-4892-ab7b-9250adef1b60 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:39.476241 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-44e86249-e39d-4892-ab7b-9250adef1b60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:39.477708 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3123c585-b41e-44ae-8a4b-f1c403070d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:39.478895 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3123c585-b41e-44ae-8a4b-f1c403070d26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:39.479174 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3123c585-b41e-44ae-8a4b-f1c403070d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:39.479442 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3123c585-b41e-44ae-8a4b-f1c403070d26 admin admin] PATH_INFO: `/v3/users/4d2b1093bbee49f2ba251fbda56cd41c` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:39.480612 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3123c585-b41e-44ae-8a4b-f1c403070d26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d2b1093bbee49f2ba251fbda56cd41c)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:39.482558 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3123c585-b41e-44ae-8a4b-f1c403070d26 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:39.489044 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e86249-e39d-4892-ab7b-9250adef1b60 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:39.495351 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e53551-66d8-4012-9221-4d08d92d9cd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:39.495990 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e53551-66d8-4012-9221-4d08d92d9cd9 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:39.496274 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e53551-66d8-4012-9221-4d08d92d9cd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:39.496553 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e53551-66d8-4012-9221-4d08d92d9cd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:39.497195 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e53551-66d8-4012-9221-4d08d92d9cd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:39.498969 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e53551-66d8-4012-9221-4d08d92d9cd9 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:39.509491 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1020/2032] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:39:39 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 07:39:39.513756 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e53551-66d8-4012-9221-4d08d92d9cd9 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:39.523442 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e53551-66d8-4012-9221-4d08d92d9cd9 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:39.528511 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a34f04f-520f-46af-8646-0f1f36f6475e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:39.529105 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a34f04f-520f-46af-8646-0f1f36f6475e admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:39.529382 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a34f04f-520f-46af-8646-0f1f36f6475e admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:39.529759 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a34f04f-520f-46af-8646-0f1f36f6475e admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:39.530392 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a34f04f-520f-46af-8646-0f1f36f6475e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:39.531965 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a34f04f-520f-46af-8646-0f1f36f6475e admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:39.543926 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1021/2033] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:39:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 07:39:39.548251 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a34f04f-520f-46af-8646-0f1f36f6475e admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:39.558989 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a34f04f-520f-46af-8646-0f1f36f6475e admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:39.563624 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1992246-1bbb-4e55-9f3e-f105e914a4a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:39.564277 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1992246-1bbb-4e55-9f3e-f105e914a4a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:39.564613 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1992246-1bbb-4e55-9f3e-f105e914a4a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:39.564839 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1992246-1bbb-4e55-9f3e-f105e914a4a9 admin admin] PATH_INFO: `/v3/projects/06390074fe824d749abd5ebf04604e48/users/71857fabfb1249f0be41e512ed63cb2f/roles/053e12af84e943e699460a9d76c2072b` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:39.579906 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1992246-1bbb-4e55-9f3e-f105e914a4a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06390074fe824d749abd5ebf04604e48, user_id=71857fabfb1249f0be41e512ed63cb2f, role_id=053e12af84e943e699460a9d76c2072b)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:39.582637 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1992246-1bbb-4e55-9f3e-f105e914a4a9 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:39.665059 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1022/2034] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:39:39 2022] PUT /identity/v3/projects/06390074fe824d749abd5ebf04604e48/users/71857fabfb1249f0be41e512ed63cb2f/roles/053e12af84e943e699460a9d76c2072b => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 07:39:39.675759 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e99d3ee-46ea-4af2-b420-b8ff3399718d None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:39.676050 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e99d3ee-46ea-4af2-b420-b8ff3399718d None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:39.676326 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e99d3ee-46ea-4af2-b420-b8ff3399718d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:39.705800 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-3e99d3ee-46ea-4af2-b420-b8ff3399718d None None] MFA Rules not processed for user `71857fabfb1249f0be41e512ed63cb2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:39:39.706999 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-3e99d3ee-46ea-4af2-b420-b8ff3399718d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:39.730282 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-3123c585-b41e-44ae-8a4b-f1c403070d26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d2b1093bbee49f2ba251fbda56cd41c'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:39.730953 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-3123c585-b41e-44ae-8a4b-f1c403070d26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d2b1093bbee49f2ba251fbda56cd41c'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:39.777705 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1023/2035] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:39:39 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 07:39:39.789530 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e99d3ee-46ea-4af2-b420-b8ff3399718d None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:39.791530 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-3e99d3ee-46ea-4af2-b420-b8ff3399718d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:39.800594 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-3123c585-b41e-44ae-8a4b-f1c403070d26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d2b1093bbee49f2ba251fbda56cd41c'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:39.801382 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-3123c585-b41e-44ae-8a4b-f1c403070d26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d2b1093bbee49f2ba251fbda56cd41c'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:39.809426 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-3123c585-b41e-44ae-8a4b-f1c403070d26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d2b1093bbee49f2ba251fbda56cd41c'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:39.827673 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1013/2036] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:39:39 2022] DELETE /identity/v3/users/4d2b1093bbee49f2ba251fbda56cd41c => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 07:39:39.872381 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e99d3ee-46ea-4af2-b420-b8ff3399718d None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:39.922516 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46804eb-9e1a-469a-98c0-e1e675db8716 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37438b032a1f46d3bdf7121ad0cffbc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:39.923061 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46804eb-9e1a-469a-98c0-e1e675db8716 service glance] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:39.923344 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46804eb-9e1a-469a-98c0-e1e675db8716 service glance] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:39.923623 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46804eb-9e1a-469a-98c0-e1e675db8716 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:39.925408 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-f46804eb-9e1a-469a-98c0-e1e675db8716 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:39.983634 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46804eb-9e1a-469a-98c0-e1e675db8716 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:39.985082 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46804eb-9e1a-469a-98c0-e1e675db8716 service glance] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:39.994204 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1024/2037] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:39:39 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 07:39:40.050359 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:40.233137 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3123c585-b41e-44ae-8a4b-f1c403070d26 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:40.235347 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-3123c585-b41e-44ae-8a4b-f1c403070d26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:40.307537 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3123c585-b41e-44ae-8a4b-f1c403070d26 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:40.312739 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e9c334-f615-4807-b707-726481e85428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:40.313940 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e9c334-f615-4807-b707-726481e85428 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:40.314230 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e9c334-f615-4807-b707-726481e85428 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:40.314512 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e9c334-f615-4807-b707-726481e85428 admin admin] PATH_INFO: `/v3/projects/ae7ab4f8df2d4c0582d72a66d5854848` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:40.316091 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e9c334-f615-4807-b707-726481e85428 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae7ab4f8df2d4c0582d72a66d5854848)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:40.318554 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e9c334-f615-4807-b707-726481e85428 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:40.341156 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:40.434565 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-06e9c334-f615-4807-b707-726481e85428 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae7ab4f8df2d4c0582d72a66d5854848', 'request_id': 'req-06e9c334-f615-4807-b707-726481e85428'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:40.440834 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-06e9c334-f615-4807-b707-726481e85428 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae7ab4f8df2d4c0582d72a66d5854848', 'request_id': 'req-06e9c334-f615-4807-b707-726481e85428'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:40.552290 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-06e9c334-f615-4807-b707-726481e85428 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae7ab4f8df2d4c0582d72a66d5854848', 'request_id': 'req-06e9c334-f615-4807-b707-726481e85428'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:40.563624 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-06e9c334-f615-4807-b707-726481e85428 admin admin] The token cache is being invalidate because project ae7ab4f8df2d4c0582d72a66d5854848 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:39:40.563917 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-06e9c334-f615-4807-b707-726481e85428 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:40.564558 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-06e9c334-f615-4807-b707-726481e85428 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:40.566124 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1014/2038] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:39:40 2022] DELETE /identity/v3/projects/ae7ab4f8df2d4c0582d72a66d5854848 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:39:40.574257 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df5687d-a842-457f-96db-bb6119b74dc9 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:40.574838 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df5687d-a842-457f-96db-bb6119b74dc9 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:40.574838 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df5687d-a842-457f-96db-bb6119b74dc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:40.596877 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-5df5687d-a842-457f-96db-bb6119b74dc9 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:39:40.598098 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-5df5687d-a842-457f-96db-bb6119b74dc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:40.659725 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1025/2039] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:39:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:39:40.667450 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ebd836-57ce-4100-9cdc-de355114b451 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:40.667813 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ebd836-57ce-4100-9cdc-de355114b451 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:40.668156 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ebd836-57ce-4100-9cdc-de355114b451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:40.690516 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-e2ebd836-57ce-4100-9cdc-de355114b451 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:39:40.691657 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-e2ebd836-57ce-4100-9cdc-de355114b451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:40.700317 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1015/2040] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:39:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:39:40.704461 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df5687d-a842-457f-96db-bb6119b74dc9 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:40.716009 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df5687d-a842-457f-96db-bb6119b74dc9 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:40.720680 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29245d6-457a-4b6f-a4c4-775dad955024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:40.721280 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29245d6-457a-4b6f-a4c4-775dad955024 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:40.721560 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29245d6-457a-4b6f-a4c4-775dad955024 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:40.722025 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29245d6-457a-4b6f-a4c4-775dad955024 admin admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:40.722485 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29245d6-457a-4b6f-a4c4-775dad955024 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:39:40.722826 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29245d6-457a-4b6f-a4c4-775dad955024 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:40.724493 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29245d6-457a-4b6f-a4c4-775dad955024 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:40.736486 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1026/2041] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 15 07:39:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:39:40.747814 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ebd836-57ce-4100-9cdc-de355114b451 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:40.751430 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ebd836-57ce-4100-9cdc-de355114b451 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:40.756390 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e80aad-b1a1-4846-ab23-5946f6e4067a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:40.756992 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e80aad-b1a1-4846-ab23-5946f6e4067a admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:40.757270 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e80aad-b1a1-4846-ab23-5946f6e4067a admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:40.757542 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e80aad-b1a1-4846-ab23-5946f6e4067a admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:40.758241 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e80aad-b1a1-4846-ab23-5946f6e4067a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:40.760973 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e80aad-b1a1-4846-ab23-5946f6e4067a admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:40.864991 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1016/2042] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:39:40 2022] POST /identity/v3/projects => generated 363 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:39:40.871468 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29245d6-457a-4b6f-a4c4-775dad955024 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:40.881686 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29245d6-457a-4b6f-a4c4-775dad955024 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:40.895721 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:40.945270 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba25142d-dd5a-4cfa-b7bb-6102e53359cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:40.945795 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba25142d-dd5a-4cfa-b7bb-6102e53359cd admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:40.946083 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba25142d-dd5a-4cfa-b7bb-6102e53359cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:40.946847 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba25142d-dd5a-4cfa-b7bb-6102e53359cd admin admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:40.947591 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba25142d-dd5a-4cfa-b7bb-6102e53359cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:40.949762 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba25142d-dd5a-4cfa-b7bb-6102e53359cd admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:41.039478 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1027/2043] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:39:40 2022] POST /identity/v3/users => generated 282 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:39:41.043791 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e80aad-b1a1-4846-ab23-5946f6e4067a admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:41.053967 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e80aad-b1a1-4846-ab23-5946f6e4067a admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:41.058930 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b06bd8d-4e00-434c-a03d-b1f9b0657a7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:41.059516 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b06bd8d-4e00-434c-a03d-b1f9b0657a7f admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:41.059838 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b06bd8d-4e00-434c-a03d-b1f9b0657a7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:41.060072 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b06bd8d-4e00-434c-a03d-b1f9b0657a7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:41.060734 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b06bd8d-4e00-434c-a03d-b1f9b0657a7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:41.062471 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b06bd8d-4e00-434c-a03d-b1f9b0657a7f admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:41.072960 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1017/2044] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:39:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:39:41.076696 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba25142d-dd5a-4cfa-b7bb-6102e53359cd admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:41.086746 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba25142d-dd5a-4cfa-b7bb-6102e53359cd admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:41.092086 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e9655c-119f-413e-a21b-f23083950aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:41.092681 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e9655c-119f-413e-a21b-f23083950aa8 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:41.092964 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e9655c-119f-413e-a21b-f23083950aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:41.093243 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e9655c-119f-413e-a21b-f23083950aa8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:41.094002 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e9655c-119f-413e-a21b-f23083950aa8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:41.095613 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e9655c-119f-413e-a21b-f23083950aa8 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:41.107592 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1028/2045] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:39:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:39:41.111842 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b06bd8d-4e00-434c-a03d-b1f9b0657a7f admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:41.122899 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b06bd8d-4e00-434c-a03d-b1f9b0657a7f admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:41.127586 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34341f9f-0509-4f99-9308-cd18bd60853a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:41.128278 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34341f9f-0509-4f99-9308-cd18bd60853a admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:41.128562 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34341f9f-0509-4f99-9308-cd18bd60853a admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:41.128839 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34341f9f-0509-4f99-9308-cd18bd60853a admin admin] PATH_INFO: `/v3/projects/2c682de488a84b4f83cb03831bea4956/users/19a76d23c3aa4a52b65852da7dcf6c8d/roles/053e12af84e943e699460a9d76c2072b` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:41.145206 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34341f9f-0509-4f99-9308-cd18bd60853a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c682de488a84b4f83cb03831bea4956, user_id=19a76d23c3aa4a52b65852da7dcf6c8d, role_id=053e12af84e943e699460a9d76c2072b)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:41.151349 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34341f9f-0509-4f99-9308-cd18bd60853a admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:41.232853 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1018/2046] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:39:41 2022] PUT /identity/v3/projects/2c682de488a84b4f83cb03831bea4956/users/19a76d23c3aa4a52b65852da7dcf6c8d/roles/053e12af84e943e699460a9d76c2072b => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:39:41.243654 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca0507-fac3-493f-b577-6455eea797cb None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:41.244531 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca0507-fac3-493f-b577-6455eea797cb None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:41.244862 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca0507-fac3-493f-b577-6455eea797cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:41.275704 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-a2ca0507-fac3-493f-b577-6455eea797cb None None] MFA Rules not processed for user `19a76d23c3aa4a52b65852da7dcf6c8d`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:39:41.276702 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-a2ca0507-fac3-493f-b577-6455eea797cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:41.344413 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1029/2047] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:39:41 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:39:41.349048 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34341f9f-0509-4f99-9308-cd18bd60853a admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:41.414392 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34341f9f-0509-4f99-9308-cd18bd60853a admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:41.419513 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d04282e-fffd-4f8b-928f-83f4acb79f2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:41.420118 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d04282e-fffd-4f8b-928f-83f4acb79f2d admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:41.420389 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d04282e-fffd-4f8b-928f-83f4acb79f2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:41.420663 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d04282e-fffd-4f8b-928f-83f4acb79f2d admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:41.421302 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d04282e-fffd-4f8b-928f-83f4acb79f2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:41.423231 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d04282e-fffd-4f8b-928f-83f4acb79f2d admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:41.539583 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1019/2048] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:39:41 2022] POST /identity/v3/projects => generated 365 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 07:39:41.544201 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ca0507-fac3-493f-b577-6455eea797cb None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:41.555614 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ca0507-fac3-493f-b577-6455eea797cb None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:41.629569 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2df253-9e53-401a-8fab-d628e4948a5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:41.630513 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2df253-9e53-401a-8fab-d628e4948a5d admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:41.630840 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2df253-9e53-401a-8fab-d628e4948a5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:41.631137 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2df253-9e53-401a-8fab-d628e4948a5d admin admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:41.632353 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2df253-9e53-401a-8fab-d628e4948a5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:41.636637 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2df253-9e53-401a-8fab-d628e4948a5d admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:41.751930 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1030/2049] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:39:41 2022] POST /identity/v3/users => generated 289 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:39:41.756025 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d04282e-fffd-4f8b-928f-83f4acb79f2d admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:41.770966 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d04282e-fffd-4f8b-928f-83f4acb79f2d admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:41.776194 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0579b23c-3793-4d75-bf45-8842ae82ae94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:41.777012 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0579b23c-3793-4d75-bf45-8842ae82ae94 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:41.777145 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0579b23c-3793-4d75-bf45-8842ae82ae94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:41.777419 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0579b23c-3793-4d75-bf45-8842ae82ae94 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:41.778104 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0579b23c-3793-4d75-bf45-8842ae82ae94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:41.779804 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0579b23c-3793-4d75-bf45-8842ae82ae94 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:41.794431 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1020/2050] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:39:41 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:39:41.798449 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2df253-9e53-401a-8fab-d628e4948a5d admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:41.810455 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2df253-9e53-401a-8fab-d628e4948a5d admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:41.819432 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3644748a-0a6e-4c1d-92e0-1013e61ad7d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:41.820426 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3644748a-0a6e-4c1d-92e0-1013e61ad7d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:41.820527 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3644748a-0a6e-4c1d-92e0-1013e61ad7d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:41.821004 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3644748a-0a6e-4c1d-92e0-1013e61ad7d8 admin admin] PATH_INFO: `/v3/projects/92c69126a4bc49f6a83d65769d766488/users/8cf0ce8e7c384a60a88a48e1c2024756/roles/6bbb643e0d8d4bc0a9b27c26edf59216` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:41.850920 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:41.854999 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3644748a-0a6e-4c1d-92e0-1013e61ad7d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92c69126a4bc49f6a83d65769d766488, user_id=8cf0ce8e7c384a60a88a48e1c2024756, role_id=6bbb643e0d8d4bc0a9b27c26edf59216)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:41.857936 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3644748a-0a6e-4c1d-92e0-1013e61ad7d8 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:41.977805 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1031/2051] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:39:41 2022] PUT /identity/v3/projects/92c69126a4bc49f6a83d65769d766488/users/8cf0ce8e7c384a60a88a48e1c2024756/roles/6bbb643e0d8d4bc0a9b27c26edf59216 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:39:41.977921 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0579b23c-3793-4d75-bf45-8842ae82ae94 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:41.987829 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0579b23c-3793-4d75-bf45-8842ae82ae94 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:42.001056 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:42.049105 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc06a251-f10e-4d82-a6a8-a347eec5ed83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:42.049756 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc06a251-f10e-4d82-a6a8-a347eec5ed83 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:42.050036 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc06a251-f10e-4d82-a6a8-a347eec5ed83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:42.050330 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc06a251-f10e-4d82-a6a8-a347eec5ed83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:42.051029 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc06a251-f10e-4d82-a6a8-a347eec5ed83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:42.052689 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc06a251-f10e-4d82-a6a8-a347eec5ed83 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:42.062937 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1021/2052] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:39:41 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 07:39:42.067339 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3644748a-0a6e-4c1d-92e0-1013e61ad7d8 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:42.079055 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3644748a-0a6e-4c1d-92e0-1013e61ad7d8 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:42.083978 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6083b873-70e9-4c62-9f1f-d9bc6713c5fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:42.084649 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6083b873-70e9-4c62-9f1f-d9bc6713c5fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:42.084931 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6083b873-70e9-4c62-9f1f-d9bc6713c5fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:42.085207 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6083b873-70e9-4c62-9f1f-d9bc6713c5fd admin admin] PATH_INFO: `/v3/projects/92c69126a4bc49f6a83d65769d766488/users/8cf0ce8e7c384a60a88a48e1c2024756/roles/6bbb643e0d8d4bc0a9b27c26edf59216` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:42.087145 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6083b873-70e9-4c62-9f1f-d9bc6713c5fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=92c69126a4bc49f6a83d65769d766488, user_id=8cf0ce8e7c384a60a88a48e1c2024756, role_id=6bbb643e0d8d4bc0a9b27c26edf59216)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:42.090182 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6083b873-70e9-4c62-9f1f-d9bc6713c5fd admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:42.101122 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1032/2053] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:39:42 2022] PUT /identity/v3/projects/92c69126a4bc49f6a83d65769d766488/users/8cf0ce8e7c384a60a88a48e1c2024756/roles/6bbb643e0d8d4bc0a9b27c26edf59216 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:39:42.112994 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19675e4-16c0-4cd8-a08f-451cf3472b41 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:42.113287 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19675e4-16c0-4cd8-a08f-451cf3472b41 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:42.113567 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19675e4-16c0-4cd8-a08f-451cf3472b41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:42.146408 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-e19675e4-16c0-4cd8-a08f-451cf3472b41 None None] MFA Rules not processed for user `8cf0ce8e7c384a60a88a48e1c2024756`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:39:42.147647 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-e19675e4-16c0-4cd8-a08f-451cf3472b41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:42.225321 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1022/2054] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:39:42 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:39:42.252382 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6083b873-70e9-4c62-9f1f-d9bc6713c5fd admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:42.254304 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-6083b873-70e9-4c62-9f1f-d9bc6713c5fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:42.319049 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6083b873-70e9-4c62-9f1f-d9bc6713c5fd admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:42.323986 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783e8a09-64ed-474e-91db-b8d14d92e1cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecf6642b7b84d3c9756bb41ab3349d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:42.324488 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e8a09-64ed-474e-91db-b8d14d92e1cb service cinder] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:42.324769 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e8a09-64ed-474e-91db-b8d14d92e1cb service cinder] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:42.325047 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e8a09-64ed-474e-91db-b8d14d92e1cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:42.334828 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783e8a09-64ed-474e-91db-b8d14d92e1cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:42.336312 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783e8a09-64ed-474e-91db-b8d14d92e1cb service cinder] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:42.347540 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1033/2055] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:39:42 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:39:42.964190 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19675e4-16c0-4cd8-a08f-451cf3472b41 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:42.973855 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19675e4-16c0-4cd8-a08f-451cf3472b41 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:42.977313 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2865dd7-ec69-4ce8-a66d-81a838f0e31c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecf6642b7b84d3c9756bb41ab3349d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:42.977955 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2865dd7-ec69-4ce8-a66d-81a838f0e31c service cinder] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:42.978235 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2865dd7-ec69-4ce8-a66d-81a838f0e31c service cinder] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:42.978507 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2865dd7-ec69-4ce8-a66d-81a838f0e31c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:43.042081 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2865dd7-ec69-4ce8-a66d-81a838f0e31c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:43.043815 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2865dd7-ec69-4ce8-a66d-81a838f0e31c service cinder] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:43.055780 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1023/2056] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:39:42 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 07:39:43.559891 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783e8a09-64ed-474e-91db-b8d14d92e1cb service cinder] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:43.561363 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-783e8a09-64ed-474e-91db-b8d14d92e1cb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:43.666282 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783e8a09-64ed-474e-91db-b8d14d92e1cb service cinder] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:43.682539 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390379bd-8ff0-436a-b1c1-e4c52dce98e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:43.682539 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390379bd-8ff0-436a-b1c1-e4c52dce98e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:43.682539 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390379bd-8ff0-436a-b1c1-e4c52dce98e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:43.682539 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390379bd-8ff0-436a-b1c1-e4c52dce98e9 admin admin] PATH_INFO: `/v3/users/bd4a548a417945519b5293738391c676` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:43.683194 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390379bd-8ff0-436a-b1c1-e4c52dce98e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd4a548a417945519b5293738391c676)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:43.683194 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390379bd-8ff0-436a-b1c1-e4c52dce98e9 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:43.902142 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-390379bd-8ff0-436a-b1c1-e4c52dce98e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd4a548a417945519b5293738391c676'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:43.996431 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-390379bd-8ff0-436a-b1c1-e4c52dce98e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd4a548a417945519b5293738391c676'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:43.997122 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-390379bd-8ff0-436a-b1c1-e4c52dce98e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd4a548a417945519b5293738391c676'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:43.997807 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-390379bd-8ff0-436a-b1c1-e4c52dce98e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd4a548a417945519b5293738391c676'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:44.005842 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-390379bd-8ff0-436a-b1c1-e4c52dce98e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd4a548a417945519b5293738391c676'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:44.020215 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:44.024603 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1034/2057] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:39:43 2022] DELETE /identity/v3/users/bd4a548a417945519b5293738391c676 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:39:44.029909 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9897bd5-712d-4291-bc7e-e91aff8a1f2b None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:44.030916 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9897bd5-712d-4291-bc7e-e91aff8a1f2b None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:44.034977 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9897bd5-712d-4291-bc7e-e91aff8a1f2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:44.064558 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-b9897bd5-712d-4291-bc7e-e91aff8a1f2b None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:39:44.066181 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-b9897bd5-712d-4291-bc7e-e91aff8a1f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:44.115879 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:44.148756 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1024/2058] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:39:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:39:44.159515 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390379bd-8ff0-436a-b1c1-e4c52dce98e9 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:44.163998 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-390379bd-8ff0-436a-b1c1-e4c52dce98e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:44.229334 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390379bd-8ff0-436a-b1c1-e4c52dce98e9 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:44.233108 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1de4150-586a-4ab1-8ad8-29c3ac5134c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:44.236757 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1de4150-586a-4ab1-8ad8-29c3ac5134c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:44.236757 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1de4150-586a-4ab1-8ad8-29c3ac5134c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:44.236757 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1de4150-586a-4ab1-8ad8-29c3ac5134c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:44.244509 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1de4150-586a-4ab1-8ad8-29c3ac5134c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:44.246013 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1de4150-586a-4ab1-8ad8-29c3ac5134c1 service neutron] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:44.255998 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1035/2059] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:39:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:39:44.715190 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9897bd5-712d-4291-bc7e-e91aff8a1f2b None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:44.718538 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-b9897bd5-712d-4291-bc7e-e91aff8a1f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:44.731500 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9897bd5-712d-4291-bc7e-e91aff8a1f2b None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:44.742079 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d507389b-6c42-42a7-8464-386756ef21c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:44.742756 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d507389b-6c42-42a7-8464-386756ef21c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:44.743042 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d507389b-6c42-42a7-8464-386756ef21c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:44.743328 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d507389b-6c42-42a7-8464-386756ef21c5 admin admin] PATH_INFO: `/v3/projects/e913941efffd4e1a9ac45c05a0a9b086` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:44.748128 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d507389b-6c42-42a7-8464-386756ef21c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e913941efffd4e1a9ac45c05a0a9b086)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:44.750142 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d507389b-6c42-42a7-8464-386756ef21c5 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:44.856987 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-d507389b-6c42-42a7-8464-386756ef21c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e913941efffd4e1a9ac45c05a0a9b086', 'request_id': 'req-d507389b-6c42-42a7-8464-386756ef21c5'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:44.863286 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-d507389b-6c42-42a7-8464-386756ef21c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e913941efffd4e1a9ac45c05a0a9b086', 'request_id': 'req-d507389b-6c42-42a7-8464-386756ef21c5'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:44.962045 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-d507389b-6c42-42a7-8464-386756ef21c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e913941efffd4e1a9ac45c05a0a9b086', 'request_id': 'req-d507389b-6c42-42a7-8464-386756ef21c5'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:44.974571 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-d507389b-6c42-42a7-8464-386756ef21c5 admin admin] The token cache is being invalidate because project e913941efffd4e1a9ac45c05a0a9b086 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:39:44.974862 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-d507389b-6c42-42a7-8464-386756ef21c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:44.975491 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-d507389b-6c42-42a7-8464-386756ef21c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:44.977039 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1025/2060] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:39:44 2022] DELETE /identity/v3/projects/e913941efffd4e1a9ac45c05a0a9b086 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:39:44.981476 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1de4150-586a-4ab1-8ad8-29c3ac5134c1 service neutron] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:44.993100 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1de4150-586a-4ab1-8ad8-29c3ac5134c1 service neutron] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:45.057806 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995b3fdf-be31-49cc-993d-8a7ddb109b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:45.058510 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995b3fdf-be31-49cc-993d-8a7ddb109b31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:45.058786 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995b3fdf-be31-49cc-993d-8a7ddb109b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:45.059063 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995b3fdf-be31-49cc-993d-8a7ddb109b31 admin admin] PATH_INFO: `/v3/users/13a80e491b31441980abeed766bcb1de` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:45.060298 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995b3fdf-be31-49cc-993d-8a7ddb109b31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13a80e491b31441980abeed766bcb1de)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:45.062425 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995b3fdf-be31-49cc-993d-8a7ddb109b31 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:45.283903 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-995b3fdf-be31-49cc-993d-8a7ddb109b31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13a80e491b31441980abeed766bcb1de'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:45.344945 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-995b3fdf-be31-49cc-993d-8a7ddb109b31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13a80e491b31441980abeed766bcb1de'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:45.345632 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-995b3fdf-be31-49cc-993d-8a7ddb109b31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13a80e491b31441980abeed766bcb1de'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:45.346257 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-995b3fdf-be31-49cc-993d-8a7ddb109b31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13a80e491b31441980abeed766bcb1de'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:45.354397 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-995b3fdf-be31-49cc-993d-8a7ddb109b31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13a80e491b31441980abeed766bcb1de'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:45.372964 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1036/2061] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:39:44 2022] DELETE /identity/v3/users/13a80e491b31441980abeed766bcb1de => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:39:45.895738 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d507389b-6c42-42a7-8464-386756ef21c5 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:45.897594 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-d507389b-6c42-42a7-8464-386756ef21c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:45.919624 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d507389b-6c42-42a7-8464-386756ef21c5 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:45.987350 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35059219-7192-45ed-ac09-35f8b238ac24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:45.987984 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35059219-7192-45ed-ac09-35f8b238ac24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:45.988266 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35059219-7192-45ed-ac09-35f8b238ac24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:45.988541 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35059219-7192-45ed-ac09-35f8b238ac24 admin admin] PATH_INFO: `/v3/projects/9321b7dd6ea642bf8cf77cc164a4f0b8` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:45.989714 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35059219-7192-45ed-ac09-35f8b238ac24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9321b7dd6ea642bf8cf77cc164a4f0b8)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:45.991642 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35059219-7192-45ed-ac09-35f8b238ac24 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:46.128350 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-35059219-7192-45ed-ac09-35f8b238ac24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9321b7dd6ea642bf8cf77cc164a4f0b8', 'request_id': 'req-35059219-7192-45ed-ac09-35f8b238ac24'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:46.134692 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-35059219-7192-45ed-ac09-35f8b238ac24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9321b7dd6ea642bf8cf77cc164a4f0b8', 'request_id': 'req-35059219-7192-45ed-ac09-35f8b238ac24'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:46.201671 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-35059219-7192-45ed-ac09-35f8b238ac24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9321b7dd6ea642bf8cf77cc164a4f0b8', 'request_id': 'req-35059219-7192-45ed-ac09-35f8b238ac24'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:46.203819 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:46.212710 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-35059219-7192-45ed-ac09-35f8b238ac24 admin admin] The token cache is being invalidate because project 9321b7dd6ea642bf8cf77cc164a4f0b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:39:46.214957 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-35059219-7192-45ed-ac09-35f8b238ac24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:46.214957 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-35059219-7192-45ed-ac09-35f8b238ac24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:46.215386 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1026/2062] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:39:45 2022] DELETE /identity/v3/projects/9321b7dd6ea642bf8cf77cc164a4f0b8 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:39:46.219478 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995b3fdf-be31-49cc-993d-8a7ddb109b31 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:46.220921 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-995b3fdf-be31-49cc-993d-8a7ddb109b31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:46.234426 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995b3fdf-be31-49cc-993d-8a7ddb109b31 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:46.253219 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:46.292694 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6048f7-4fd3-4bcb-b35c-ed0c9623f5ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:46.293333 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6048f7-4fd3-4bcb-b35c-ed0c9623f5ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:46.293666 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6048f7-4fd3-4bcb-b35c-ed0c9623f5ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:46.293986 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6048f7-4fd3-4bcb-b35c-ed0c9623f5ae admin admin] PATH_INFO: `/v3/users/878d1ce4c716423ba2d33e35c30c178e` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:46.295113 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6048f7-4fd3-4bcb-b35c-ed0c9623f5ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=878d1ce4c716423ba2d33e35c30c178e)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:46.297074 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6048f7-4fd3-4bcb-b35c-ed0c9623f5ae admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:46.479210 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-de6048f7-4fd3-4bcb-b35c-ed0c9623f5ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '878d1ce4c716423ba2d33e35c30c178e'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:46.538940 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-de6048f7-4fd3-4bcb-b35c-ed0c9623f5ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '878d1ce4c716423ba2d33e35c30c178e'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:46.539592 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-de6048f7-4fd3-4bcb-b35c-ed0c9623f5ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '878d1ce4c716423ba2d33e35c30c178e'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:46.540222 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-de6048f7-4fd3-4bcb-b35c-ed0c9623f5ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '878d1ce4c716423ba2d33e35c30c178e'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:46.548636 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-de6048f7-4fd3-4bcb-b35c-ed0c9623f5ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '878d1ce4c716423ba2d33e35c30c178e'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:46.573142 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1037/2063] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:39:46 2022] DELETE /identity/v3/users/878d1ce4c716423ba2d33e35c30c178e => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:39:46.718392 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35059219-7192-45ed-ac09-35f8b238ac24 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:46.720068 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-35059219-7192-45ed-ac09-35f8b238ac24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:46.734225 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35059219-7192-45ed-ac09-35f8b238ac24 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:46.804438 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88f8968-9f58-4d7b-93dc-c22b8ef9ac33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:46.805709 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88f8968-9f58-4d7b-93dc-c22b8ef9ac33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:46.806307 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88f8968-9f58-4d7b-93dc-c22b8ef9ac33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:46.807406 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88f8968-9f58-4d7b-93dc-c22b8ef9ac33 admin admin] PATH_INFO: `/v3/users/19a76d23c3aa4a52b65852da7dcf6c8d` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:46.809506 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88f8968-9f58-4d7b-93dc-c22b8ef9ac33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19a76d23c3aa4a52b65852da7dcf6c8d)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:46.813422 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88f8968-9f58-4d7b-93dc-c22b8ef9ac33 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:46.984554 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-a88f8968-9f58-4d7b-93dc-c22b8ef9ac33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19a76d23c3aa4a52b65852da7dcf6c8d'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:46.985276 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-a88f8968-9f58-4d7b-93dc-c22b8ef9ac33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19a76d23c3aa4a52b65852da7dcf6c8d'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:47.066497 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6048f7-4fd3-4bcb-b35c-ed0c9623f5ae admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:47.067909 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-de6048f7-4fd3-4bcb-b35c-ed0c9623f5ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:47.070840 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-a88f8968-9f58-4d7b-93dc-c22b8ef9ac33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19a76d23c3aa4a52b65852da7dcf6c8d'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:47.071629 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-a88f8968-9f58-4d7b-93dc-c22b8ef9ac33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19a76d23c3aa4a52b65852da7dcf6c8d'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:47.080724 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-a88f8968-9f58-4d7b-93dc-c22b8ef9ac33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19a76d23c3aa4a52b65852da7dcf6c8d'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:47.086598 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6048f7-4fd3-4bcb-b35c-ed0c9623f5ae admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:47.092117 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5599742-a4e7-4021-bc01-a9a5179adf0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:47.092745 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5599742-a4e7-4021-bc01-a9a5179adf0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:47.093027 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5599742-a4e7-4021-bc01-a9a5179adf0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:47.093307 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5599742-a4e7-4021-bc01-a9a5179adf0d admin admin] PATH_INFO: `/v3/projects/42fb97e9fb2f4b12a563d161a6bec6cc` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:47.094480 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5599742-a4e7-4021-bc01-a9a5179adf0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=42fb97e9fb2f4b12a563d161a6bec6cc)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:47.096823 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5599742-a4e7-4021-bc01-a9a5179adf0d admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:47.100723 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1027/2064] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:39:46 2022] DELETE /identity/v3/users/19a76d23c3aa4a52b65852da7dcf6c8d => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:39:47.105675 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1a4e59-1a31-4b68-89d5-0fcaf61002fb None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:47.105964 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1a4e59-1a31-4b68-89d5-0fcaf61002fb None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:47.106253 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1a4e59-1a31-4b68-89d5-0fcaf61002fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:47.126881 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-8a1a4e59-1a31-4b68-89d5-0fcaf61002fb None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:39:47.128125 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-8a1a4e59-1a31-4b68-89d5-0fcaf61002fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:47.187974 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1028/2065] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:39:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 07:39:47.197768 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1a4e59-1a31-4b68-89d5-0fcaf61002fb None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:47.199219 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-8a1a4e59-1a31-4b68-89d5-0fcaf61002fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:47.212454 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:47.229336 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-c5599742-a4e7-4021-bc01-a9a5179adf0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42fb97e9fb2f4b12a563d161a6bec6cc', 'request_id': 'req-c5599742-a4e7-4021-bc01-a9a5179adf0d'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:47.232782 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-c5599742-a4e7-4021-bc01-a9a5179adf0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42fb97e9fb2f4b12a563d161a6bec6cc', 'request_id': 'req-c5599742-a4e7-4021-bc01-a9a5179adf0d'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:47.257996 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1a4e59-1a31-4b68-89d5-0fcaf61002fb None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:47.304736 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-c5599742-a4e7-4021-bc01-a9a5179adf0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42fb97e9fb2f4b12a563d161a6bec6cc', 'request_id': 'req-c5599742-a4e7-4021-bc01-a9a5179adf0d'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:47.307651 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e497f92-6861-4b46-bc74-1a5768e6f9f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:47.308263 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e497f92-6861-4b46-bc74-1a5768e6f9f4 service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:47.308581 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e497f92-6861-4b46-bc74-1a5768e6f9f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:47.308811 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e497f92-6861-4b46-bc74-1a5768e6f9f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:47.318745 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-c5599742-a4e7-4021-bc01-a9a5179adf0d admin admin] The token cache is being invalidate because project 42fb97e9fb2f4b12a563d161a6bec6cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:39:47.319102 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-c5599742-a4e7-4021-bc01-a9a5179adf0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:47.320094 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-c5599742-a4e7-4021-bc01-a9a5179adf0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:47.326751 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1038/2066] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:39:47 2022] DELETE /identity/v3/projects/42fb97e9fb2f4b12a563d161a6bec6cc => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 07:39:47.328051 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c86ac8-7cab-4a9e-87e2-ea04891c0c3a None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:47.328340 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c86ac8-7cab-4a9e-87e2-ea04891c0c3a None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:47.328614 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c86ac8-7cab-4a9e-87e2-ea04891c0c3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:47.348717 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-b5c86ac8-7cab-4a9e-87e2-ea04891c0c3a None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:39:47.349994 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-b5c86ac8-7cab-4a9e-87e2-ea04891c0c3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:47.373370 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:47.377846 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e497f92-6861-4b46-bc74-1a5768e6f9f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:47.379271 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e497f92-6861-4b46-bc74-1a5768e6f9f4 service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:47.388906 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1029/2067] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:39:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 07:39:47.416934 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1039/2068] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:39:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:39:47.426308 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3d5e6a-6350-4bec-a163-3de677690aab None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:47.426609 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3d5e6a-6350-4bec-a163-3de677690aab None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:47.426897 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3d5e6a-6350-4bec-a163-3de677690aab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:47.448104 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-4e3d5e6a-6350-4bec-a163-3de677690aab None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:39:47.449523 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-4e3d5e6a-6350-4bec-a163-3de677690aab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:47.457689 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1030/2069] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:39:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:39:47.462841 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c86ac8-7cab-4a9e-87e2-ea04891c0c3a None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:47.474010 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c86ac8-7cab-4a9e-87e2-ea04891c0c3a None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:47.479062 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721b63ee-e377-4b6d-bb38-9e51f96ed4ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:47.479870 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721b63ee-e377-4b6d-bb38-9e51f96ed4ce admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:47.480159 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721b63ee-e377-4b6d-bb38-9e51f96ed4ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:47.480624 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721b63ee-e377-4b6d-bb38-9e51f96ed4ce admin admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:47.481297 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721b63ee-e377-4b6d-bb38-9e51f96ed4ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:39:47.481633 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721b63ee-e377-4b6d-bb38-9e51f96ed4ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:47.483305 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721b63ee-e377-4b6d-bb38-9e51f96ed4ce admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:47.501617 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1040/2070] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 15 07:39:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:39:47.503275 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3d5e6a-6350-4bec-a163-3de677690aab None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:47.513653 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3d5e6a-6350-4bec-a163-3de677690aab None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:47.518674 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfe5adb-7679-4418-b4cd-88c027a5f20f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:47.523509 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe5adb-7679-4418-b4cd-88c027a5f20f admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:47.523509 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe5adb-7679-4418-b4cd-88c027a5f20f admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:47.523509 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe5adb-7679-4418-b4cd-88c027a5f20f admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:47.523509 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfe5adb-7679-4418-b4cd-88c027a5f20f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:47.523509 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfe5adb-7679-4418-b4cd-88c027a5f20f admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:47.644419 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1031/2071] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:39:47 2022] POST /identity/v3/projects => generated 383 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:39:47.645792 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721b63ee-e377-4b6d-bb38-9e51f96ed4ce admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:47.656745 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721b63ee-e377-4b6d-bb38-9e51f96ed4ce admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:47.716028 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff410b7-3131-424e-bbee-060c94d68620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:47.716648 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff410b7-3131-424e-bbee-060c94d68620 admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:47.716994 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff410b7-3131-424e-bbee-060c94d68620 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:47.717204 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff410b7-3131-424e-bbee-060c94d68620 admin admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:47.717971 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff410b7-3131-424e-bbee-060c94d68620 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:47.719818 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff410b7-3131-424e-bbee-060c94d68620 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:47.815259 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1041/2072] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:39:47 2022] POST /identity/v3/users => generated 292 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:39:47.822916 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfe5adb-7679-4418-b4cd-88c027a5f20f admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:47.822916 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-5bfe5adb-7679-4418-b4cd-88c027a5f20f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:47.826126 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff410b7-3131-424e-bbee-060c94d68620 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:47.839607 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff410b7-3131-424e-bbee-060c94d68620 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:47.845835 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfe5adb-7679-4418-b4cd-88c027a5f20f admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:47.847925 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e8d1a8-b08c-44ae-99ed-b950cdd4d30a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:47.848598 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e8d1a8-b08c-44ae-99ed-b950cdd4d30a admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:47.848886 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e8d1a8-b08c-44ae-99ed-b950cdd4d30a admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:47.849160 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e8d1a8-b08c-44ae-99ed-b950cdd4d30a admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:47.849916 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e8d1a8-b08c-44ae-99ed-b950cdd4d30a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:47.857206 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9da7fd-3d14-4ba5-b3c1-7fbc1771cfed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:47.857768 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e8d1a8-b08c-44ae-99ed-b950cdd4d30a admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:47.858164 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9da7fd-3d14-4ba5-b3c1-7fbc1771cfed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:47.858164 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9da7fd-3d14-4ba5-b3c1-7fbc1771cfed admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:47.858164 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9da7fd-3d14-4ba5-b3c1-7fbc1771cfed admin admin] PATH_INFO: `/v3/projects/2c682de488a84b4f83cb03831bea4956` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:47.858164 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9da7fd-3d14-4ba5-b3c1-7fbc1771cfed admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c682de488a84b4f83cb03831bea4956)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:47.858164 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9da7fd-3d14-4ba5-b3c1-7fbc1771cfed admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:47.865853 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1042/2073] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:39:47 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:39:47.870307 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e8d1a8-b08c-44ae-99ed-b950cdd4d30a admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:47.884651 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e8d1a8-b08c-44ae-99ed-b950cdd4d30a admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:47.890285 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696f5abf-a2dd-4b53-a2f9-2a38bfa83196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:47.890877 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696f5abf-a2dd-4b53-a2f9-2a38bfa83196 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:47.891158 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696f5abf-a2dd-4b53-a2f9-2a38bfa83196 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:47.891431 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696f5abf-a2dd-4b53-a2f9-2a38bfa83196 admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:47.892850 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696f5abf-a2dd-4b53-a2f9-2a38bfa83196 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:47.894535 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696f5abf-a2dd-4b53-a2f9-2a38bfa83196 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:47.905498 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1043/2074] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:39:47 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:39:47.909763 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696f5abf-a2dd-4b53-a2f9-2a38bfa83196 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:47.920318 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696f5abf-a2dd-4b53-a2f9-2a38bfa83196 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:47.925021 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23422205-1bed-4d97-98b6-ed2aa0ebd33c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:47.925713 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23422205-1bed-4d97-98b6-ed2aa0ebd33c admin admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:47.926044 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23422205-1bed-4d97-98b6-ed2aa0ebd33c admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:47.926383 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23422205-1bed-4d97-98b6-ed2aa0ebd33c admin admin] PATH_INFO: `/v3/projects/747defe4045f4ea7a52c5311add3d342/users/679274c61fb44a15b009345131814959/roles/053e12af84e943e699460a9d76c2072b` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:47.946653 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23422205-1bed-4d97-98b6-ed2aa0ebd33c admin admin] RBAC: Authorizing `identity:create_grant(project_id=747defe4045f4ea7a52c5311add3d342, user_id=679274c61fb44a15b009345131814959, role_id=053e12af84e943e699460a9d76c2072b)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:47.949405 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23422205-1bed-4d97-98b6-ed2aa0ebd33c admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:48.022131 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1044/2075] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:39:47 2022] PUT /identity/v3/projects/747defe4045f4ea7a52c5311add3d342/users/679274c61fb44a15b009345131814959/roles/053e12af84e943e699460a9d76c2072b => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 07:39:48.035844 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc326706-1bf4-4bb3-aaf6-2f8efa8205f1 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:48.036132 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc326706-1bf4-4bb3-aaf6-2f8efa8205f1 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:48.036405 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc326706-1bf4-4bb3-aaf6-2f8efa8205f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:48.039669 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:48.044734 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-3c9da7fd-3d14-4ba5-b3c1-7fbc1771cfed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c682de488a84b4f83cb03831bea4956', 'request_id': 'req-3c9da7fd-3d14-4ba5-b3c1-7fbc1771cfed'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:48.058088 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-3c9da7fd-3d14-4ba5-b3c1-7fbc1771cfed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c682de488a84b4f83cb03831bea4956', 'request_id': 'req-3c9da7fd-3d14-4ba5-b3c1-7fbc1771cfed'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:48.070883 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-bc326706-1bf4-4bb3-aaf6-2f8efa8205f1 None None] MFA Rules not processed for user `679274c61fb44a15b009345131814959`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:39:48.072130 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-bc326706-1bf4-4bb3-aaf6-2f8efa8205f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:48.122072 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:48.129977 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-3c9da7fd-3d14-4ba5-b3c1-7fbc1771cfed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c682de488a84b4f83cb03831bea4956', 'request_id': 'req-3c9da7fd-3d14-4ba5-b3c1-7fbc1771cfed'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:48.142977 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-3c9da7fd-3d14-4ba5-b3c1-7fbc1771cfed admin admin] The token cache is being invalidate because project 2c682de488a84b4f83cb03831bea4956 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:39:48.143302 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-3c9da7fd-3d14-4ba5-b3c1-7fbc1771cfed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:48.145355 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-3c9da7fd-3d14-4ba5-b3c1-7fbc1771cfed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:48.145472 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1032/2076] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:39:47 2022] DELETE /identity/v3/projects/2c682de488a84b4f83cb03831bea4956 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:39:48.146001 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1045/2077] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:39:48 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 07:39:48.149285 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9da7fd-3d14-4ba5-b3c1-7fbc1771cfed admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:48.150055 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc326706-1bf4-4bb3-aaf6-2f8efa8205f1 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:48.151501 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-3c9da7fd-3d14-4ba5-b3c1-7fbc1771cfed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:48.216791 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc326706-1bf4-4bb3-aaf6-2f8efa8205f1 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:48.219961 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9da7fd-3d14-4ba5-b3c1-7fbc1771cfed admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:48.221910 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2928892-20d8-46ed-8dbc-64891b43939b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:48.230552 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2928892-20d8-46ed-8dbc-64891b43939b admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:48.230552 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2928892-20d8-46ed-8dbc-64891b43939b admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:48.230552 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2928892-20d8-46ed-8dbc-64891b43939b admin admin] PATH_INFO: `/v3/projects` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:48.230552 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2928892-20d8-46ed-8dbc-64891b43939b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:48.231348 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c27e4f-7f15-466b-8d7d-d44550cc3e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:48.231457 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2928892-20d8-46ed-8dbc-64891b43939b admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:48.231544 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c27e4f-7f15-466b-8d7d-d44550cc3e56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:48.231544 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c27e4f-7f15-466b-8d7d-d44550cc3e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:48.231544 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c27e4f-7f15-466b-8d7d-d44550cc3e56 admin admin] PATH_INFO: `/v3/users/8cf0ce8e7c384a60a88a48e1c2024756` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:48.231544 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c27e4f-7f15-466b-8d7d-d44550cc3e56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8cf0ce8e7c384a60a88a48e1c2024756)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:48.231544 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c27e4f-7f15-466b-8d7d-d44550cc3e56 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:48.318107 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1046/2078] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:39:48 2022] POST /identity/v3/projects => generated 383 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 07:39:48.321981 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2928892-20d8-46ed-8dbc-64891b43939b admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:48.324138 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-d2928892-20d8-46ed-8dbc-64891b43939b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:48.393225 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2928892-20d8-46ed-8dbc-64891b43939b admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:48.398383 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a0d4c0-8a69-414f-bd39-a39ab8e72dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:48.399966 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a0d4c0-8a69-414f-bd39-a39ab8e72dbd admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:48.399966 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a0d4c0-8a69-414f-bd39-a39ab8e72dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:48.399966 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a0d4c0-8a69-414f-bd39-a39ab8e72dbd admin admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:48.400214 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a0d4c0-8a69-414f-bd39-a39ab8e72dbd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:48.402561 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a0d4c0-8a69-414f-bd39-a39ab8e72dbd admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:48.435018 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-94c27e4f-7f15-466b-8d7d-d44550cc3e56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8cf0ce8e7c384a60a88a48e1c2024756'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:48.435839 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-94c27e4f-7f15-466b-8d7d-d44550cc3e56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8cf0ce8e7c384a60a88a48e1c2024756'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:48.517056 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1047/2079] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:39:48 2022] POST /identity/v3/users => generated 298 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 07:39:48.518675 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a0d4c0-8a69-414f-bd39-a39ab8e72dbd admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:48.529568 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a0d4c0-8a69-414f-bd39-a39ab8e72dbd admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:48.536311 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-94c27e4f-7f15-466b-8d7d-d44550cc3e56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8cf0ce8e7c384a60a88a48e1c2024756'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:48.536939 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-94c27e4f-7f15-466b-8d7d-d44550cc3e56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8cf0ce8e7c384a60a88a48e1c2024756'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:48.547299 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d792daa9-154c-45f9-b99c-c5513e3870f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:48.547622 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-94c27e4f-7f15-466b-8d7d-d44550cc3e56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8cf0ce8e7c384a60a88a48e1c2024756'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:48.547939 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d792daa9-154c-45f9-b99c-c5513e3870f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:48.548258 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d792daa9-154c-45f9-b99c-c5513e3870f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:48.548500 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d792daa9-154c-45f9-b99c-c5513e3870f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:48.549167 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d792daa9-154c-45f9-b99c-c5513e3870f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:48.550816 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d792daa9-154c-45f9-b99c-c5513e3870f5 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:48.565630 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1048/2080] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:39:48 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 07:39:48.568578 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d792daa9-154c-45f9-b99c-c5513e3870f5 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:48.570480 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-d792daa9-154c-45f9-b99c-c5513e3870f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:48.587703 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1033/2081] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:39:48 2022] DELETE /identity/v3/users/8cf0ce8e7c384a60a88a48e1c2024756 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:39:48.600849 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d792daa9-154c-45f9-b99c-c5513e3870f5 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:48.681365 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:48.694619 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93976e71-5137-433b-8e15-dc52ebccc313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:48.696039 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93976e71-5137-433b-8e15-dc52ebccc313 admin admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:48.696039 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93976e71-5137-433b-8e15-dc52ebccc313 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:48.696039 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93976e71-5137-433b-8e15-dc52ebccc313 admin admin] PATH_INFO: `/v3/projects/08b5ae83a3a5483994e878bc1d5b382e/users/ab6540a7c535407fa2fcb7d36c47b059/roles/6bbb643e0d8d4bc0a9b27c26edf59216` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:48.724557 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93976e71-5137-433b-8e15-dc52ebccc313 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08b5ae83a3a5483994e878bc1d5b382e, user_id=ab6540a7c535407fa2fcb7d36c47b059, role_id=6bbb643e0d8d4bc0a9b27c26edf59216)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:48.725467 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93976e71-5137-433b-8e15-dc52ebccc313 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:48.830493 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1049/2082] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:39:48 2022] PUT /identity/v3/projects/08b5ae83a3a5483994e878bc1d5b382e/users/ab6540a7c535407fa2fcb7d36c47b059/roles/6bbb643e0d8d4bc0a9b27c26edf59216 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 07:39:48.831828 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c27e4f-7f15-466b-8d7d-d44550cc3e56 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:48.842806 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c27e4f-7f15-466b-8d7d-d44550cc3e56 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:48.904809 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:48.914616 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea9a6d5-e892-4265-8afd-1dbfbc643ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:48.915296 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea9a6d5-e892-4265-8afd-1dbfbc643ee7 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:48.915511 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea9a6d5-e892-4265-8afd-1dbfbc643ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:48.915786 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea9a6d5-e892-4265-8afd-1dbfbc643ee7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:48.917003 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea9a6d5-e892-4265-8afd-1dbfbc643ee7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:48.918639 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea9a6d5-e892-4265-8afd-1dbfbc643ee7 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:48.929281 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1034/2083] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:39:48 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:39:48.934111 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93976e71-5137-433b-8e15-dc52ebccc313 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:48.951515 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93976e71-5137-433b-8e15-dc52ebccc313 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:48.956678 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e959ec8-9d26-40d1-9e2d-e745f567002f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:48.957695 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e959ec8-9d26-40d1-9e2d-e745f567002f admin admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:48.957917 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e959ec8-9d26-40d1-9e2d-e745f567002f admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:48.961790 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e959ec8-9d26-40d1-9e2d-e745f567002f admin admin] PATH_INFO: `/v3/projects/08b5ae83a3a5483994e878bc1d5b382e/users/ab6540a7c535407fa2fcb7d36c47b059/roles/6bbb643e0d8d4bc0a9b27c26edf59216` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:48.961790 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e959ec8-9d26-40d1-9e2d-e745f567002f admin admin] RBAC: Authorizing `identity:create_grant(project_id=08b5ae83a3a5483994e878bc1d5b382e, user_id=ab6540a7c535407fa2fcb7d36c47b059, role_id=6bbb643e0d8d4bc0a9b27c26edf59216)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:48.963165 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e959ec8-9d26-40d1-9e2d-e745f567002f admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:48.974297 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1050/2084] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:39:48 2022] PUT /identity/v3/projects/08b5ae83a3a5483994e878bc1d5b382e/users/ab6540a7c535407fa2fcb7d36c47b059/roles/6bbb643e0d8d4bc0a9b27c26edf59216 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:39:48.987382 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89cba37-aff3-4c48-b31b-1c2bd2571c87 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:48.987382 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89cba37-aff3-4c48-b31b-1c2bd2571c87 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:48.987382 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89cba37-aff3-4c48-b31b-1c2bd2571c87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:49.027716 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-c89cba37-aff3-4c48-b31b-1c2bd2571c87 None None] MFA Rules not processed for user `ab6540a7c535407fa2fcb7d36c47b059`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:39:49.028726 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-c89cba37-aff3-4c48-b31b-1c2bd2571c87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:49.124937 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1035/2085] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:39:48 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:39:49.141561 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e959ec8-9d26-40d1-9e2d-e745f567002f admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:49.143630 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-1e959ec8-9d26-40d1-9e2d-e745f567002f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:49.245133 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e959ec8-9d26-40d1-9e2d-e745f567002f admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:49.249123 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e972612-3451-4ccd-8f34-fc7526cfcf1d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecf6642b7b84d3c9756bb41ab3349d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:49.249848 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e972612-3451-4ccd-8f34-fc7526cfcf1d service cinder] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:49.250291 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e972612-3451-4ccd-8f34-fc7526cfcf1d service cinder] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:49.250564 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e972612-3451-4ccd-8f34-fc7526cfcf1d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:49.267607 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e972612-3451-4ccd-8f34-fc7526cfcf1d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:49.267607 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e972612-3451-4ccd-8f34-fc7526cfcf1d service cinder] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:49.281635 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1051/2086] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:39:49 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:39:49.298490 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89cba37-aff3-4c48-b31b-1c2bd2571c87 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:49.299763 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-c89cba37-aff3-4c48-b31b-1c2bd2571c87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:49.429299 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89cba37-aff3-4c48-b31b-1c2bd2571c87 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:49.434801 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3bf7be-c3a3-4bd6-9a79-df56be209743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:49.435446 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3bf7be-c3a3-4bd6-9a79-df56be209743 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:49.435732 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3bf7be-c3a3-4bd6-9a79-df56be209743 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:49.436012 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3bf7be-c3a3-4bd6-9a79-df56be209743 admin admin] PATH_INFO: `/v3/projects/92c69126a4bc49f6a83d65769d766488` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:49.437160 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3bf7be-c3a3-4bd6-9a79-df56be209743 admin admin] RBAC: Authorizing `identity:delete_project(project_id=92c69126a4bc49f6a83d65769d766488)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:49.439626 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3bf7be-c3a3-4bd6-9a79-df56be209743 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:49.550986 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-8c3bf7be-c3a3-4bd6-9a79-df56be209743 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92c69126a4bc49f6a83d65769d766488', 'request_id': 'req-8c3bf7be-c3a3-4bd6-9a79-df56be209743'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:49.558357 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-8c3bf7be-c3a3-4bd6-9a79-df56be209743 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92c69126a4bc49f6a83d65769d766488', 'request_id': 'req-8c3bf7be-c3a3-4bd6-9a79-df56be209743'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:49.653783 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-8c3bf7be-c3a3-4bd6-9a79-df56be209743 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92c69126a4bc49f6a83d65769d766488', 'request_id': 'req-8c3bf7be-c3a3-4bd6-9a79-df56be209743'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:49.665056 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-8c3bf7be-c3a3-4bd6-9a79-df56be209743 admin admin] The token cache is being invalidate because project 92c69126a4bc49f6a83d65769d766488 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:39:49.665354 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-8c3bf7be-c3a3-4bd6-9a79-df56be209743 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:49.666020 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-8c3bf7be-c3a3-4bd6-9a79-df56be209743 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:49.667871 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1036/2087] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:39:49 2022] DELETE /identity/v3/projects/92c69126a4bc49f6a83d65769d766488 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:39:49.673707 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b52d023-6b3c-4049-9975-e12e4516384f None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:49.674017 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b52d023-6b3c-4049-9975-e12e4516384f None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:49.674310 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b52d023-6b3c-4049-9975-e12e4516384f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:49.694730 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-1b52d023-6b3c-4049-9975-e12e4516384f None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:39:49.695934 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-1b52d023-6b3c-4049-9975-e12e4516384f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:49.776920 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158651a8-e2ae-45a8-8562-af366b5c6b6a None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:49.777215 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158651a8-e2ae-45a8-8562-af366b5c6b6a None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:49.777501 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158651a8-e2ae-45a8-8562-af366b5c6b6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:49.784118 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1052/2088] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:39:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 07:39:49.817755 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b52d023-6b3c-4049-9975-e12e4516384f None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:49.819467 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-1b52d023-6b3c-4049-9975-e12e4516384f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:49.831867 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-158651a8-e2ae-45a8-8562-af366b5c6b6a None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:39:49.833167 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-158651a8-e2ae-45a8-8562-af366b5c6b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:49.843643 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1037/2089] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:39:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:39:49.847838 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158651a8-e2ae-45a8-8562-af366b5c6b6a None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:49.857689 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158651a8-e2ae-45a8-8562-af366b5c6b6a None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:49.862403 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed108510-0a4c-4af5-9c4b-6692bd242648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:49.862941 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed108510-0a4c-4af5-9c4b-6692bd242648 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:49.863221 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed108510-0a4c-4af5-9c4b-6692bd242648 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:49.863492 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed108510-0a4c-4af5-9c4b-6692bd242648 admin admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:49.864142 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed108510-0a4c-4af5-9c4b-6692bd242648 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:39:49.864444 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed108510-0a4c-4af5-9c4b-6692bd242648 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:49.866192 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed108510-0a4c-4af5-9c4b-6692bd242648 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:49.883328 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1038/2090] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 15 07:39:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:39:49.884398 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed108510-0a4c-4af5-9c4b-6692bd242648 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:49.892521 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b52d023-6b3c-4049-9975-e12e4516384f None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:49.894764 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed108510-0a4c-4af5-9c4b-6692bd242648 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:49.896313 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63b3c79-40d4-4410-b100-e88f2102d3e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecf6642b7b84d3c9756bb41ab3349d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:49.896932 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63b3c79-40d4-4410-b100-e88f2102d3e5 service cinder] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:49.897216 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63b3c79-40d4-4410-b100-e88f2102d3e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:49.897491 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63b3c79-40d4-4410-b100-e88f2102d3e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:49.899302 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-b63b3c79-40d4-4410-b100-e88f2102d3e5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:49.899729 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c057204-b735-4640-aa90-634ab073338f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:49.900343 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c057204-b735-4640-aa90-634ab073338f admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:49.900651 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c057204-b735-4640-aa90-634ab073338f admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:49.901217 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c057204-b735-4640-aa90-634ab073338f admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:49.901914 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c057204-b735-4640-aa90-634ab073338f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:49.904007 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c057204-b735-4640-aa90-634ab073338f admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:49.915203 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:49.930370 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:49.998863 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63b3c79-40d4-4410-b100-e88f2102d3e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:50.000314 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63b3c79-40d4-4410-b100-e88f2102d3e5 service cinder] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:50.012065 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1053/2091] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:39:49 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 07:39:50.023783 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1039/2092] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:39:49 2022] POST /identity/v3/projects => generated 389 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 07:39:50.027063 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63b3c79-40d4-4410-b100-e88f2102d3e5 service cinder] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:50.041754 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63b3c79-40d4-4410-b100-e88f2102d3e5 service cinder] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:50.130752 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5082e595-3a8a-4216-b436-3e5e08a1b842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:50.131522 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5082e595-3a8a-4216-b436-3e5e08a1b842 admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:50.133830 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5082e595-3a8a-4216-b436-3e5e08a1b842 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:50.133830 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5082e595-3a8a-4216-b436-3e5e08a1b842 admin admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:50.133830 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5082e595-3a8a-4216-b436-3e5e08a1b842 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:50.153105 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5082e595-3a8a-4216-b436-3e5e08a1b842 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:50.269540 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1054/2093] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:39:50 2022] POST /identity/v3/users => generated 295 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:39:50.284311 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c057204-b735-4640-aa90-634ab073338f admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:50.297864 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c057204-b735-4640-aa90-634ab073338f admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:50.302601 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463a97fb-9715-44d7-9591-e0205b66bcf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:50.303219 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463a97fb-9715-44d7-9591-e0205b66bcf2 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:50.303503 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463a97fb-9715-44d7-9591-e0205b66bcf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:50.303782 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463a97fb-9715-44d7-9591-e0205b66bcf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:50.304431 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463a97fb-9715-44d7-9591-e0205b66bcf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:50.306108 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463a97fb-9715-44d7-9591-e0205b66bcf2 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:50.317061 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1040/2094] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:39:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:39:50.321629 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5082e595-3a8a-4216-b436-3e5e08a1b842 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:50.334016 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5082e595-3a8a-4216-b436-3e5e08a1b842 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:50.339651 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fd7d44-ddda-49d5-b818-1b1f43d979e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:50.340260 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fd7d44-ddda-49d5-b818-1b1f43d979e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:50.340538 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fd7d44-ddda-49d5-b818-1b1f43d979e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:50.340811 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fd7d44-ddda-49d5-b818-1b1f43d979e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:50.341450 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fd7d44-ddda-49d5-b818-1b1f43d979e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:50.343077 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fd7d44-ddda-49d5-b818-1b1f43d979e4 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:50.353257 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1055/2095] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:39:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:39:50.358490 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463a97fb-9715-44d7-9591-e0205b66bcf2 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:50.368480 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463a97fb-9715-44d7-9591-e0205b66bcf2 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:50.373177 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d837f8-9d69-425d-b8a8-1174a84edbab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:50.374015 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d837f8-9d69-425d-b8a8-1174a84edbab admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:50.378333 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d837f8-9d69-425d-b8a8-1174a84edbab admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:50.378333 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d837f8-9d69-425d-b8a8-1174a84edbab admin admin] PATH_INFO: `/v3/projects/aa7cf3839c6447149563d50b6362296b/users/14a9b112724b4a058baabe03f409bd64/roles/053e12af84e943e699460a9d76c2072b` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:50.389645 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d837f8-9d69-425d-b8a8-1174a84edbab admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa7cf3839c6447149563d50b6362296b, user_id=14a9b112724b4a058baabe03f409bd64, role_id=053e12af84e943e699460a9d76c2072b)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:50.392290 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d837f8-9d69-425d-b8a8-1174a84edbab admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:50.480050 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1041/2096] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:39:50 2022] PUT /identity/v3/projects/aa7cf3839c6447149563d50b6362296b/users/14a9b112724b4a058baabe03f409bd64/roles/053e12af84e943e699460a9d76c2072b => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:39:50.491705 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ef3331-bdf4-4b4f-b36c-0eee89867b36 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:50.492022 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ef3331-bdf4-4b4f-b36c-0eee89867b36 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:50.492272 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ef3331-bdf4-4b4f-b36c-0eee89867b36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:50.531698 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-b8ef3331-bdf4-4b4f-b36c-0eee89867b36 None None] MFA Rules not processed for user `14a9b112724b4a058baabe03f409bd64`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:39:50.534029 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-b8ef3331-bdf4-4b4f-b36c-0eee89867b36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:50.606582 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1056/2097] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:39:50 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:39:50.611218 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d837f8-9d69-425d-b8a8-1174a84edbab admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:50.678396 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d837f8-9d69-425d-b8a8-1174a84edbab admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:50.683696 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbbb91d-131f-467a-ab4e-5b06833dad85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:50.684306 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbbb91d-131f-467a-ab4e-5b06833dad85 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:50.684591 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbbb91d-131f-467a-ab4e-5b06833dad85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:50.684861 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbbb91d-131f-467a-ab4e-5b06833dad85 admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:50.685490 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbbb91d-131f-467a-ab4e-5b06833dad85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:50.687503 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbbb91d-131f-467a-ab4e-5b06833dad85 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:50.841952 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1042/2098] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:39:50 2022] POST /identity/v3/projects => generated 389 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:39:50.848519 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ef3331-bdf4-4b4f-b36c-0eee89867b36 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:50.859598 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ef3331-bdf4-4b4f-b36c-0eee89867b36 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:50.896644 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:50.950378 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ae4625-0694-4c91-8a41-b3af04dcc07e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:50.951024 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ae4625-0694-4c91-8a41-b3af04dcc07e admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:50.951309 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ae4625-0694-4c91-8a41-b3af04dcc07e admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:50.951583 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ae4625-0694-4c91-8a41-b3af04dcc07e admin admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:50.952252 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ae4625-0694-4c91-8a41-b3af04dcc07e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:50.954325 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ae4625-0694-4c91-8a41-b3af04dcc07e admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:51.046332 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1057/2099] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:39:50 2022] POST /identity/v3/users => generated 301 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:39:51.050777 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbbb91d-131f-467a-ab4e-5b06833dad85 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:51.062064 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbbb91d-131f-467a-ab4e-5b06833dad85 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:51.067037 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a84ef7-58ba-4906-a309-abc03b3c8db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:51.067849 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a84ef7-58ba-4906-a309-abc03b3c8db3 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:51.068084 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a84ef7-58ba-4906-a309-abc03b3c8db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:51.068369 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a84ef7-58ba-4906-a309-abc03b3c8db3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:51.069080 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a84ef7-58ba-4906-a309-abc03b3c8db3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:51.070746 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a84ef7-58ba-4906-a309-abc03b3c8db3 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:51.085614 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1043/2100] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:39:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 07:39:51.087068 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ae4625-0694-4c91-8a41-b3af04dcc07e admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:51.097912 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ae4625-0694-4c91-8a41-b3af04dcc07e admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:51.102542 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa81462-6b5c-49b4-b335-9a0b27eba57e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:51.103373 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa81462-6b5c-49b4-b335-9a0b27eba57e admin admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:51.103475 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa81462-6b5c-49b4-b335-9a0b27eba57e admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:51.104370 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa81462-6b5c-49b4-b335-9a0b27eba57e admin admin] PATH_INFO: `/v3/projects/8d1e06b1642641759634b0534fde0899/users/a24ae70a6f93424db41c4082761313ad/roles/6bbb643e0d8d4bc0a9b27c26edf59216` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:51.121451 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa81462-6b5c-49b4-b335-9a0b27eba57e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d1e06b1642641759634b0534fde0899, user_id=a24ae70a6f93424db41c4082761313ad, role_id=6bbb643e0d8d4bc0a9b27c26edf59216)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:51.124670 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa81462-6b5c-49b4-b335-9a0b27eba57e admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:51.210394 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1058/2101] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:39:51 2022] PUT /identity/v3/projects/8d1e06b1642641759634b0534fde0899/users/a24ae70a6f93424db41c4082761313ad/roles/6bbb643e0d8d4bc0a9b27c26edf59216 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:39:51.214464 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a84ef7-58ba-4906-a309-abc03b3c8db3 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:51.224413 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a84ef7-58ba-4906-a309-abc03b3c8db3 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:51.287586 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe70698-cf1a-48fc-bfca-f912322b3ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:51.288239 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe70698-cf1a-48fc-bfca-f912322b3ec4 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:51.288497 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe70698-cf1a-48fc-bfca-f912322b3ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:51.288767 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe70698-cf1a-48fc-bfca-f912322b3ec4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:51.289407 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe70698-cf1a-48fc-bfca-f912322b3ec4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:51.291019 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe70698-cf1a-48fc-bfca-f912322b3ec4 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:51.292337 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:51.302287 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1044/2102] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:39:51 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:39:51.305882 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa81462-6b5c-49b4-b335-9a0b27eba57e admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:51.316958 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa81462-6b5c-49b4-b335-9a0b27eba57e admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:51.322162 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd0a0fb-d413-407e-9920-b94bac2c71b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:51.323427 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd0a0fb-d413-407e-9920-b94bac2c71b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:51.323783 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd0a0fb-d413-407e-9920-b94bac2c71b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:51.324101 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd0a0fb-d413-407e-9920-b94bac2c71b6 admin admin] PATH_INFO: `/v3/projects/8d1e06b1642641759634b0534fde0899/users/a24ae70a6f93424db41c4082761313ad/roles/6bbb643e0d8d4bc0a9b27c26edf59216` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:51.326136 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd0a0fb-d413-407e-9920-b94bac2c71b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d1e06b1642641759634b0534fde0899, user_id=a24ae70a6f93424db41c4082761313ad, role_id=6bbb643e0d8d4bc0a9b27c26edf59216)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:51.332813 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd0a0fb-d413-407e-9920-b94bac2c71b6 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:51.339775 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1059/2103] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:39:51 2022] PUT /identity/v3/projects/8d1e06b1642641759634b0534fde0899/users/a24ae70a6f93424db41c4082761313ad/roles/6bbb643e0d8d4bc0a9b27c26edf59216 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:39:51.350084 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc408d2-99fc-4c2d-a0ca-18f27a6da4cf None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:51.350376 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc408d2-99fc-4c2d-a0ca-18f27a6da4cf None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:51.350709 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc408d2-99fc-4c2d-a0ca-18f27a6da4cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:51.379431 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-efc408d2-99fc-4c2d-a0ca-18f27a6da4cf None None] MFA Rules not processed for user `a24ae70a6f93424db41c4082761313ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:39:51.382109 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-efc408d2-99fc-4c2d-a0ca-18f27a6da4cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:51.453611 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1045/2104] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:39:51 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:39:51.464844 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd0a0fb-d413-407e-9920-b94bac2c71b6 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:51.466680 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-5cd0a0fb-d413-407e-9920-b94bac2c71b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:51.529404 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd0a0fb-d413-407e-9920-b94bac2c71b6 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:51.533227 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263c585f-3478-45ee-96a1-31e192dcb98d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecf6642b7b84d3c9756bb41ab3349d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:51.533874 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263c585f-3478-45ee-96a1-31e192dcb98d service cinder] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:51.534163 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263c585f-3478-45ee-96a1-31e192dcb98d service cinder] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:51.534438 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263c585f-3478-45ee-96a1-31e192dcb98d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:51.544105 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263c585f-3478-45ee-96a1-31e192dcb98d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:51.545646 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263c585f-3478-45ee-96a1-31e192dcb98d service cinder] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:51.555569 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1060/2105] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:39:51 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:39:51.693784 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc408d2-99fc-4c2d-a0ca-18f27a6da4cf None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:51.761133 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc408d2-99fc-4c2d-a0ca-18f27a6da4cf None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:51.765664 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43625896-0f19-4b34-a559-07e2bd13946e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:51.766298 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43625896-0f19-4b34-a559-07e2bd13946e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:51.766576 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43625896-0f19-4b34-a559-07e2bd13946e admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:51.766849 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43625896-0f19-4b34-a559-07e2bd13946e admin admin] PATH_INFO: `/v3/users/14a9b112724b4a058baabe03f409bd64` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:51.767996 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43625896-0f19-4b34-a559-07e2bd13946e admin admin] RBAC: Authorizing `identity:delete_user(user_id=14a9b112724b4a058baabe03f409bd64)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:51.769924 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43625896-0f19-4b34-a559-07e2bd13946e admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:51.968018 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-43625896-0f19-4b34-a559-07e2bd13946e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14a9b112724b4a058baabe03f409bd64'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:51.968668 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-43625896-0f19-4b34-a559-07e2bd13946e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14a9b112724b4a058baabe03f409bd64'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:52.036022 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-43625896-0f19-4b34-a559-07e2bd13946e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14a9b112724b4a058baabe03f409bd64'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:52.036643 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-43625896-0f19-4b34-a559-07e2bd13946e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14a9b112724b4a058baabe03f409bd64'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:52.044953 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-43625896-0f19-4b34-a559-07e2bd13946e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14a9b112724b4a058baabe03f409bd64'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:52.058800 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:52.063036 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1046/2106] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:39:51 2022] DELETE /identity/v3/users/14a9b112724b4a058baabe03f409bd64 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:39:52.068385 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00704535-6caf-4f5f-8a50-d048de1401b3 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:52.068955 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00704535-6caf-4f5f-8a50-d048de1401b3 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:52.069539 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00704535-6caf-4f5f-8a50-d048de1401b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:52.092929 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-00704535-6caf-4f5f-8a50-d048de1401b3 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:39:52.094164 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-00704535-6caf-4f5f-8a50-d048de1401b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:52.152993 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1061/2107] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:39:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:39:52.162658 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43625896-0f19-4b34-a559-07e2bd13946e admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:52.164103 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-43625896-0f19-4b34-a559-07e2bd13946e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:52.222436 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43625896-0f19-4b34-a559-07e2bd13946e admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:52.225943 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc68446-f3f8-4915-a199-9b20498e838f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:52.226544 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc68446-f3f8-4915-a199-9b20498e838f service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:52.226822 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc68446-f3f8-4915-a199-9b20498e838f service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:52.227091 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc68446-f3f8-4915-a199-9b20498e838f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:52.236009 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc68446-f3f8-4915-a199-9b20498e838f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:52.237413 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc68446-f3f8-4915-a199-9b20498e838f service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:52.246735 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1047/2108] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:39:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:39:52.826779 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00704535-6caf-4f5f-8a50-d048de1401b3 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:52.828344 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-00704535-6caf-4f5f-8a50-d048de1401b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:52.841032 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00704535-6caf-4f5f-8a50-d048de1401b3 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:52.846520 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37c0030-6b27-4094-87e7-677657379a85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:52.846919 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37c0030-6b27-4094-87e7-677657379a85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:52.849734 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37c0030-6b27-4094-87e7-677657379a85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:52.850039 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37c0030-6b27-4094-87e7-677657379a85 admin admin] PATH_INFO: `/v3/projects/aa7cf3839c6447149563d50b6362296b` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:52.851249 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37c0030-6b27-4094-87e7-677657379a85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa7cf3839c6447149563d50b6362296b)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:52.853196 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37c0030-6b27-4094-87e7-677657379a85 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:52.871941 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:52.977472 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-e37c0030-6b27-4094-87e7-677657379a85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa7cf3839c6447149563d50b6362296b', 'request_id': 'req-e37c0030-6b27-4094-87e7-677657379a85'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:52.984391 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-e37c0030-6b27-4094-87e7-677657379a85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa7cf3839c6447149563d50b6362296b', 'request_id': 'req-e37c0030-6b27-4094-87e7-677657379a85'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:53.038903 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-e37c0030-6b27-4094-87e7-677657379a85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa7cf3839c6447149563d50b6362296b', 'request_id': 'req-e37c0030-6b27-4094-87e7-677657379a85'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:53.050632 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-e37c0030-6b27-4094-87e7-677657379a85 admin admin] The token cache is being invalidate because project aa7cf3839c6447149563d50b6362296b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:39:53.050921 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-e37c0030-6b27-4094-87e7-677657379a85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:53.051730 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-e37c0030-6b27-4094-87e7-677657379a85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:53.053563 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1062/2109] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:39:52 2022] DELETE /identity/v3/projects/aa7cf3839c6447149563d50b6362296b => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:39:53.058002 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc68446-f3f8-4915-a199-9b20498e838f service neutron] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:53.067808 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc68446-f3f8-4915-a199-9b20498e838f service neutron] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:53.123772 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f253818-992c-48a9-a5b4-3af1d0c9e117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:53.124404 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f253818-992c-48a9-a5b4-3af1d0c9e117 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:53.124692 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f253818-992c-48a9-a5b4-3af1d0c9e117 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:53.124973 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f253818-992c-48a9-a5b4-3af1d0c9e117 admin admin] PATH_INFO: `/v3/users/a24ae70a6f93424db41c4082761313ad` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:53.126125 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f253818-992c-48a9-a5b4-3af1d0c9e117 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a24ae70a6f93424db41c4082761313ad)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:53.128060 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f253818-992c-48a9-a5b4-3af1d0c9e117 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:53.307920 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-3f253818-992c-48a9-a5b4-3af1d0c9e117 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a24ae70a6f93424db41c4082761313ad'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:53.308578 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-3f253818-992c-48a9-a5b4-3af1d0c9e117 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a24ae70a6f93424db41c4082761313ad'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:53.373804 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-3f253818-992c-48a9-a5b4-3af1d0c9e117 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a24ae70a6f93424db41c4082761313ad'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:53.374441 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-3f253818-992c-48a9-a5b4-3af1d0c9e117 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a24ae70a6f93424db41c4082761313ad'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:53.382207 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-3f253818-992c-48a9-a5b4-3af1d0c9e117 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a24ae70a6f93424db41c4082761313ad'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:53.400014 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1048/2110] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:39:53 2022] DELETE /identity/v3/users/a24ae70a6f93424db41c4082761313ad => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 07:39:53.809512 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37c0030-6b27-4094-87e7-677657379a85 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:53.810983 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-e37c0030-6b27-4094-87e7-677657379a85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:53.827558 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37c0030-6b27-4094-87e7-677657379a85 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:53.898982 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196a608c-ea02-43e9-9f53-1077a375a450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:53.899217 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196a608c-ea02-43e9-9f53-1077a375a450 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:53.899576 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196a608c-ea02-43e9-9f53-1077a375a450 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:53.899937 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196a608c-ea02-43e9-9f53-1077a375a450 admin admin] PATH_INFO: `/v3/projects/8d1e06b1642641759634b0534fde0899` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:53.902515 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196a608c-ea02-43e9-9f53-1077a375a450 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d1e06b1642641759634b0534fde0899)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:53.904502 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196a608c-ea02-43e9-9f53-1077a375a450 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:54.003358 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-196a608c-ea02-43e9-9f53-1077a375a450 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d1e06b1642641759634b0534fde0899', 'request_id': 'req-196a608c-ea02-43e9-9f53-1077a375a450'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:54.009767 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-196a608c-ea02-43e9-9f53-1077a375a450 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d1e06b1642641759634b0534fde0899', 'request_id': 'req-196a608c-ea02-43e9-9f53-1077a375a450'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:54.090560 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-196a608c-ea02-43e9-9f53-1077a375a450 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d1e06b1642641759634b0534fde0899', 'request_id': 'req-196a608c-ea02-43e9-9f53-1077a375a450'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:54.101519 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-196a608c-ea02-43e9-9f53-1077a375a450 admin admin] The token cache is being invalidate because project 8d1e06b1642641759634b0534fde0899 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:39:54.104579 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-196a608c-ea02-43e9-9f53-1077a375a450 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:54.104579 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-196a608c-ea02-43e9-9f53-1077a375a450 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:54.104579 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1063/2111] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:39:53 2022] DELETE /identity/v3/projects/8d1e06b1642641759634b0534fde0899 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:39:54.110663 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c916373d-285c-4858-b292-73a840c6f119 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:54.111198 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c916373d-285c-4858-b292-73a840c6f119 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:54.111198 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c916373d-285c-4858-b292-73a840c6f119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:54.131025 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-c916373d-285c-4858-b292-73a840c6f119 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:39:54.132252 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-c916373d-285c-4858-b292-73a840c6f119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:54.149600 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:54.199657 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1049/2112] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:39:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:39:54.207890 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b59a7f7-ed48-483d-92ca-e7764f4d0578 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:54.208176 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b59a7f7-ed48-483d-92ca-e7764f4d0578 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:54.208460 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b59a7f7-ed48-483d-92ca-e7764f4d0578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:54.230889 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-1b59a7f7-ed48-483d-92ca-e7764f4d0578 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:39:54.232104 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-1b59a7f7-ed48-483d-92ca-e7764f4d0578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:54.240117 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1064/2113] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:39:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:39:54.244377 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c916373d-285c-4858-b292-73a840c6f119 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:54.254376 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c916373d-285c-4858-b292-73a840c6f119 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:54.259244 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421b24d5-2088-493f-918e-170a3e7a3218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:54.259864 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421b24d5-2088-493f-918e-170a3e7a3218 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:54.260191 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421b24d5-2088-493f-918e-170a3e7a3218 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:54.260635 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421b24d5-2088-493f-918e-170a3e7a3218 admin admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:54.261326 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421b24d5-2088-493f-918e-170a3e7a3218 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:39:54.261659 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421b24d5-2088-493f-918e-170a3e7a3218 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:54.263300 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421b24d5-2088-493f-918e-170a3e7a3218 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:54.275626 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1050/2114] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 15 07:39:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:39:54.281518 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b59a7f7-ed48-483d-92ca-e7764f4d0578 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:54.290817 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b59a7f7-ed48-483d-92ca-e7764f4d0578 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:54.296414 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a24dcc-4ae4-4ed5-a416-7d42ac78c42a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:54.297011 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a24dcc-4ae4-4ed5-a416-7d42ac78c42a admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:54.297289 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a24dcc-4ae4-4ed5-a416-7d42ac78c42a admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:54.297571 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a24dcc-4ae4-4ed5-a416-7d42ac78c42a admin admin] PATH_INFO: `/v3/projects` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:54.298222 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a24dcc-4ae4-4ed5-a416-7d42ac78c42a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:54.300111 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a24dcc-4ae4-4ed5-a416-7d42ac78c42a admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:54.432495 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1065/2115] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:39:54 2022] POST /identity/v3/projects => generated 371 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:39:54.436980 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421b24d5-2088-493f-918e-170a3e7a3218 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:54.447223 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421b24d5-2088-493f-918e-170a3e7a3218 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:54.504575 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e0575d-b190-4363-b201-235dea05e275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:54.505171 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e0575d-b190-4363-b201-235dea05e275 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:54.505451 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e0575d-b190-4363-b201-235dea05e275 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:54.505790 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e0575d-b190-4363-b201-235dea05e275 admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:54.506421 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e0575d-b190-4363-b201-235dea05e275 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:54.508308 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e0575d-b190-4363-b201-235dea05e275 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:54.657026 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1051/2116] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:39:54 2022] POST /identity/v3/users => generated 286 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:39:54.659009 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a24dcc-4ae4-4ed5-a416-7d42ac78c42a admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:54.665693 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:54.669875 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a24dcc-4ae4-4ed5-a416-7d42ac78c42a admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:54.675207 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68187f5-0973-4b43-807c-46ae0b739eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:54.676225 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68187f5-0973-4b43-807c-46ae0b739eae admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:54.676729 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68187f5-0973-4b43-807c-46ae0b739eae admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:54.677395 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68187f5-0973-4b43-807c-46ae0b739eae admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:54.678661 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68187f5-0973-4b43-807c-46ae0b739eae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:54.681714 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68187f5-0973-4b43-807c-46ae0b739eae admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:54.693000 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1066/2117] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:39:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:39:54.697346 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e0575d-b190-4363-b201-235dea05e275 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:54.709105 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e0575d-b190-4363-b201-235dea05e275 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:54.715095 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140a6104-3b75-4eea-a097-e7bfeb1a3175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:54.715756 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140a6104-3b75-4eea-a097-e7bfeb1a3175 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:54.716008 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140a6104-3b75-4eea-a097-e7bfeb1a3175 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:54.716281 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140a6104-3b75-4eea-a097-e7bfeb1a3175 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:54.717102 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140a6104-3b75-4eea-a097-e7bfeb1a3175 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:54.718629 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140a6104-3b75-4eea-a097-e7bfeb1a3175 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:54.730443 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1052/2118] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:39:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:39:54.734776 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68187f5-0973-4b43-807c-46ae0b739eae admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:54.746086 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68187f5-0973-4b43-807c-46ae0b739eae admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:54.751028 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd15c6a-84e7-41ab-8bb0-92c5b00619cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:54.751840 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd15c6a-84e7-41ab-8bb0-92c5b00619cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:54.752120 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd15c6a-84e7-41ab-8bb0-92c5b00619cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:54.752397 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd15c6a-84e7-41ab-8bb0-92c5b00619cf admin admin] PATH_INFO: `/v3/projects/725712114b314192bb4175523f7cceaf/users/d46df96e76c24d1c8f5ce42d1f946344/roles/053e12af84e943e699460a9d76c2072b` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:54.767676 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd15c6a-84e7-41ab-8bb0-92c5b00619cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=725712114b314192bb4175523f7cceaf, user_id=d46df96e76c24d1c8f5ce42d1f946344, role_id=053e12af84e943e699460a9d76c2072b)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:54.770443 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd15c6a-84e7-41ab-8bb0-92c5b00619cf admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:54.868190 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1067/2119] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:39:54 2022] PUT /identity/v3/projects/725712114b314192bb4175523f7cceaf/users/d46df96e76c24d1c8f5ce42d1f946344/roles/053e12af84e943e699460a9d76c2072b => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:39:54.880724 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641f17c6-c994-4127-95aa-0dd522382337 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:54.881043 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641f17c6-c994-4127-95aa-0dd522382337 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:54.881329 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641f17c6-c994-4127-95aa-0dd522382337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:54.913660 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-641f17c6-c994-4127-95aa-0dd522382337 None None] MFA Rules not processed for user `d46df96e76c24d1c8f5ce42d1f946344`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:39:54.914852 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-641f17c6-c994-4127-95aa-0dd522382337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:54.988586 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1053/2120] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:39:54 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:39:55.002607 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd15c6a-84e7-41ab-8bb0-92c5b00619cf admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:55.004340 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-afd15c6a-84e7-41ab-8bb0-92c5b00619cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:55.065088 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd15c6a-84e7-41ab-8bb0-92c5b00619cf admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:55.068747 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd666ebf-0300-4df1-a967-73a57cf384bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecf6642b7b84d3c9756bb41ab3349d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:55.069350 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd666ebf-0300-4df1-a967-73a57cf384bb service cinder] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:55.069654 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd666ebf-0300-4df1-a967-73a57cf384bb service cinder] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:55.069927 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd666ebf-0300-4df1-a967-73a57cf384bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:55.079603 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd666ebf-0300-4df1-a967-73a57cf384bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:55.081168 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd666ebf-0300-4df1-a967-73a57cf384bb service cinder] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:55.093242 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1068/2121] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:39:55 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:39:55.120065 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641f17c6-c994-4127-95aa-0dd522382337 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:55.156379 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:55.219916 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641f17c6-c994-4127-95aa-0dd522382337 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:55.224525 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73e7ba6-e4fe-4b4c-9986-adb4e0eb4fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:55.234555 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73e7ba6-e4fe-4b4c-9986-adb4e0eb4fe2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:55.234555 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73e7ba6-e4fe-4b4c-9986-adb4e0eb4fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:55.234555 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73e7ba6-e4fe-4b4c-9986-adb4e0eb4fe2 admin admin] PATH_INFO: `/v3/users/d46df96e76c24d1c8f5ce42d1f946344` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:55.234555 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73e7ba6-e4fe-4b4c-9986-adb4e0eb4fe2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d46df96e76c24d1c8f5ce42d1f946344)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:55.234555 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73e7ba6-e4fe-4b4c-9986-adb4e0eb4fe2 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:55.453253 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-b73e7ba6-e4fe-4b4c-9986-adb4e0eb4fe2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd46df96e76c24d1c8f5ce42d1f946344'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:55.454030 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-b73e7ba6-e4fe-4b4c-9986-adb4e0eb4fe2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd46df96e76c24d1c8f5ce42d1f946344'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:55.516635 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-b73e7ba6-e4fe-4b4c-9986-adb4e0eb4fe2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd46df96e76c24d1c8f5ce42d1f946344'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:55.517262 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-b73e7ba6-e4fe-4b4c-9986-adb4e0eb4fe2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd46df96e76c24d1c8f5ce42d1f946344'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:55.526279 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-b73e7ba6-e4fe-4b4c-9986-adb4e0eb4fe2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd46df96e76c24d1c8f5ce42d1f946344'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:55.545138 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1054/2122] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:39:55 2022] DELETE /identity/v3/users/d46df96e76c24d1c8f5ce42d1f946344 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:39:55.555884 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e186b5-845b-4f16-a90e-0da1885ec6f8 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:55.556177 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e186b5-845b-4f16-a90e-0da1885ec6f8 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:55.556523 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e186b5-845b-4f16-a90e-0da1885ec6f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:55.577840 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-47e186b5-845b-4f16-a90e-0da1885ec6f8 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:39:55.579056 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-47e186b5-845b-4f16-a90e-0da1885ec6f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:55.643364 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1069/2123] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:39:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:39:55.654071 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73e7ba6-e4fe-4b4c-9986-adb4e0eb4fe2 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:55.655561 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-b73e7ba6-e4fe-4b4c-9986-adb4e0eb4fe2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:55.715006 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73e7ba6-e4fe-4b4c-9986-adb4e0eb4fe2 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:55.718556 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c787f0-c8eb-4085-929e-6bc5469c7261 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:55.719241 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c787f0-c8eb-4085-929e-6bc5469c7261 service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:55.719526 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c787f0-c8eb-4085-929e-6bc5469c7261 service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:55.719822 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c787f0-c8eb-4085-929e-6bc5469c7261 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:55.729188 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c787f0-c8eb-4085-929e-6bc5469c7261 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:55.730577 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c787f0-c8eb-4085-929e-6bc5469c7261 service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:55.740362 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1055/2124] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:39:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:39:56.149903 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e186b5-845b-4f16-a90e-0da1885ec6f8 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:56.152986 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-47e186b5-845b-4f16-a90e-0da1885ec6f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:56.170925 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e186b5-845b-4f16-a90e-0da1885ec6f8 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:56.176148 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36f589f-75ae-4b57-8149-baf9040dbc01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:56.181118 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36f589f-75ae-4b57-8149-baf9040dbc01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:56.181118 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36f589f-75ae-4b57-8149-baf9040dbc01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:56.181118 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36f589f-75ae-4b57-8149-baf9040dbc01 admin admin] PATH_INFO: `/v3/projects/725712114b314192bb4175523f7cceaf` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:56.181118 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36f589f-75ae-4b57-8149-baf9040dbc01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=725712114b314192bb4175523f7cceaf)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:56.184468 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36f589f-75ae-4b57-8149-baf9040dbc01 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:56.338826 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-a36f589f-75ae-4b57-8149-baf9040dbc01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '725712114b314192bb4175523f7cceaf', 'request_id': 'req-a36f589f-75ae-4b57-8149-baf9040dbc01'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:56.346230 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-a36f589f-75ae-4b57-8149-baf9040dbc01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '725712114b314192bb4175523f7cceaf', 'request_id': 'req-a36f589f-75ae-4b57-8149-baf9040dbc01'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:56.404931 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-a36f589f-75ae-4b57-8149-baf9040dbc01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '725712114b314192bb4175523f7cceaf', 'request_id': 'req-a36f589f-75ae-4b57-8149-baf9040dbc01'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:56.410347 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:56.417020 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-a36f589f-75ae-4b57-8149-baf9040dbc01 admin admin] The token cache is being invalidate because project 725712114b314192bb4175523f7cceaf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:39:56.417332 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-a36f589f-75ae-4b57-8149-baf9040dbc01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:56.417995 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-a36f589f-75ae-4b57-8149-baf9040dbc01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:56.419473 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1070/2125] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:39:56 2022] DELETE /identity/v3/projects/725712114b314192bb4175523f7cceaf => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:39:56.588326 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c787f0-c8eb-4085-929e-6bc5469c7261 service neutron] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:56.590454 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-46c787f0-c8eb-4085-929e-6bc5469c7261 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:56.610973 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c787f0-c8eb-4085-929e-6bc5469c7261 service neutron] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:56.693292 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:56.728473 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4de088-4564-4271-a8b7-8239c0b1c790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:56.728473 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4de088-4564-4271-a8b7-8239c0b1c790 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:56.728473 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4de088-4564-4271-a8b7-8239c0b1c790 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:56.728473 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4de088-4564-4271-a8b7-8239c0b1c790 admin admin] PATH_INFO: `/v3/users/679274c61fb44a15b009345131814959` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:56.729483 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4de088-4564-4271-a8b7-8239c0b1c790 admin admin] RBAC: Authorizing `identity:delete_user(user_id=679274c61fb44a15b009345131814959)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:56.731195 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4de088-4564-4271-a8b7-8239c0b1c790 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:56.904417 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-1b4de088-4564-4271-a8b7-8239c0b1c790 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '679274c61fb44a15b009345131814959'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:56.905114 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-1b4de088-4564-4271-a8b7-8239c0b1c790 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '679274c61fb44a15b009345131814959'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:57.014413 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-1b4de088-4564-4271-a8b7-8239c0b1c790 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '679274c61fb44a15b009345131814959'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:57.015081 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-1b4de088-4564-4271-a8b7-8239c0b1c790 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '679274c61fb44a15b009345131814959'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:57.024035 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-1b4de088-4564-4271-a8b7-8239c0b1c790 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '679274c61fb44a15b009345131814959'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:57.048476 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1056/2126] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:39:56 2022] DELETE /identity/v3/users/679274c61fb44a15b009345131814959 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 07:39:57.053088 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bf6ecf-d454-4697-a507-7a5bd3c193d2 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:57.053490 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bf6ecf-d454-4697-a507-7a5bd3c193d2 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:57.053756 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bf6ecf-d454-4697-a507-7a5bd3c193d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:57.085168 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-e4bf6ecf-d454-4697-a507-7a5bd3c193d2 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:39:57.086853 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-e4bf6ecf-d454-4697-a507-7a5bd3c193d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:57.162146 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1071/2127] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:39:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:39:57.174057 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4de088-4564-4271-a8b7-8239c0b1c790 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:57.176147 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-1b4de088-4564-4271-a8b7-8239c0b1c790 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:57.257437 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4de088-4564-4271-a8b7-8239c0b1c790 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:57.260176 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ad4552-d988-4bad-b797-80106bbf0826 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:57.260811 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ad4552-d988-4bad-b797-80106bbf0826 service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:57.261139 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ad4552-d988-4bad-b797-80106bbf0826 service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:57.261416 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ad4552-d988-4bad-b797-80106bbf0826 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:57.272749 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ad4552-d988-4bad-b797-80106bbf0826 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:57.277991 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ad4552-d988-4bad-b797-80106bbf0826 service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:57.286400 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1057/2128] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:39:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:39:57.886659 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bf6ecf-d454-4697-a507-7a5bd3c193d2 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:57.890429 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-e4bf6ecf-d454-4697-a507-7a5bd3c193d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:57.912240 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bf6ecf-d454-4697-a507-7a5bd3c193d2 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:57.921069 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b46c4a-bea2-4e86-bd88-2ec629d60d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:57.922315 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b46c4a-bea2-4e86-bd88-2ec629d60d15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:57.922648 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b46c4a-bea2-4e86-bd88-2ec629d60d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:57.923118 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b46c4a-bea2-4e86-bd88-2ec629d60d15 admin admin] PATH_INFO: `/v3/projects/747defe4045f4ea7a52c5311add3d342` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:57.924424 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b46c4a-bea2-4e86-bd88-2ec629d60d15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=747defe4045f4ea7a52c5311add3d342)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:57.926434 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b46c4a-bea2-4e86-bd88-2ec629d60d15 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:58.062938 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-e8b46c4a-bea2-4e86-bd88-2ec629d60d15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '747defe4045f4ea7a52c5311add3d342', 'request_id': 'req-e8b46c4a-bea2-4e86-bd88-2ec629d60d15'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:58.073256 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-e8b46c4a-bea2-4e86-bd88-2ec629d60d15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '747defe4045f4ea7a52c5311add3d342', 'request_id': 'req-e8b46c4a-bea2-4e86-bd88-2ec629d60d15'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:58.150836 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-e8b46c4a-bea2-4e86-bd88-2ec629d60d15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '747defe4045f4ea7a52c5311add3d342', 'request_id': 'req-e8b46c4a-bea2-4e86-bd88-2ec629d60d15'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:58.165773 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-e8b46c4a-bea2-4e86-bd88-2ec629d60d15 admin admin] The token cache is being invalidate because project 747defe4045f4ea7a52c5311add3d342 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:39:58.166449 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-e8b46c4a-bea2-4e86-bd88-2ec629d60d15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:58.166944 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-e8b46c4a-bea2-4e86-bd88-2ec629d60d15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:58.168767 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1072/2129] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:39:57 2022] DELETE /identity/v3/projects/747defe4045f4ea7a52c5311add3d342 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:39:58.172750 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ad4552-d988-4bad-b797-80106bbf0826 service neutron] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:58.184333 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ad4552-d988-4bad-b797-80106bbf0826 service neutron] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:58.270824 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b316dc-a92e-4f5d-a250-60bec85ba0c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:58.271584 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b316dc-a92e-4f5d-a250-60bec85ba0c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:58.272167 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b316dc-a92e-4f5d-a250-60bec85ba0c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:58.272516 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b316dc-a92e-4f5d-a250-60bec85ba0c3 admin admin] PATH_INFO: `/v3/users/ab6540a7c535407fa2fcb7d36c47b059` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:58.273786 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b316dc-a92e-4f5d-a250-60bec85ba0c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab6540a7c535407fa2fcb7d36c47b059)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:58.282552 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b316dc-a92e-4f5d-a250-60bec85ba0c3 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:58.286923 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:58.476772 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-e1b316dc-a92e-4f5d-a250-60bec85ba0c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab6540a7c535407fa2fcb7d36c47b059'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:58.477485 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-e1b316dc-a92e-4f5d-a250-60bec85ba0c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab6540a7c535407fa2fcb7d36c47b059'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:58.574801 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-e1b316dc-a92e-4f5d-a250-60bec85ba0c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab6540a7c535407fa2fcb7d36c47b059'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:58.575627 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-e1b316dc-a92e-4f5d-a250-60bec85ba0c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab6540a7c535407fa2fcb7d36c47b059'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:58.583250 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-e1b316dc-a92e-4f5d-a250-60bec85ba0c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab6540a7c535407fa2fcb7d36c47b059'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:58.601664 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1058/2130] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:39:58 2022] DELETE /identity/v3/users/ab6540a7c535407fa2fcb7d36c47b059 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:39:59.116772 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b46c4a-bea2-4e86-bd88-2ec629d60d15 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:59.118387 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-e8b46c4a-bea2-4e86-bd88-2ec629d60d15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:59.141944 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b46c4a-bea2-4e86-bd88-2ec629d60d15 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:59.238925 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ebffb0-c5f4-4222-b8ed-9ae25171b696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:59.239584 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ebffb0-c5f4-4222-b8ed-9ae25171b696 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:59.239865 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ebffb0-c5f4-4222-b8ed-9ae25171b696 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:59.240150 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ebffb0-c5f4-4222-b8ed-9ae25171b696 admin admin] PATH_INFO: `/v3/projects/08b5ae83a3a5483994e878bc1d5b382e` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:59.241356 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ebffb0-c5f4-4222-b8ed-9ae25171b696 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08b5ae83a3a5483994e878bc1d5b382e)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:59.243702 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ebffb0-c5f4-4222-b8ed-9ae25171b696 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:59.280648 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:59.372624 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-62ebffb0-c5f4-4222-b8ed-9ae25171b696 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08b5ae83a3a5483994e878bc1d5b382e', 'request_id': 'req-62ebffb0-c5f4-4222-b8ed-9ae25171b696'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:59.380056 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-62ebffb0-c5f4-4222-b8ed-9ae25171b696 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08b5ae83a3a5483994e878bc1d5b382e', 'request_id': 'req-62ebffb0-c5f4-4222-b8ed-9ae25171b696'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:59.449168 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-62ebffb0-c5f4-4222-b8ed-9ae25171b696 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08b5ae83a3a5483994e878bc1d5b382e', 'request_id': 'req-62ebffb0-c5f4-4222-b8ed-9ae25171b696'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:59.467195 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-62ebffb0-c5f4-4222-b8ed-9ae25171b696 admin admin] The token cache is being invalidate because project 08b5ae83a3a5483994e878bc1d5b382e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:39:59.467501 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-62ebffb0-c5f4-4222-b8ed-9ae25171b696 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:59.470930 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-62ebffb0-c5f4-4222-b8ed-9ae25171b696 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:59.473623 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1073/2131] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:39:59 2022] DELETE /identity/v3/projects/08b5ae83a3a5483994e878bc1d5b382e => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:39:59.478513 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ce5ff0-5e93-49e5-a86a-90f62ffaaee8 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:59.479333 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ce5ff0-5e93-49e5-a86a-90f62ffaaee8 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:59.479610 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ce5ff0-5e93-49e5-a86a-90f62ffaaee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:59.503010 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-f5ce5ff0-5e93-49e5-a86a-90f62ffaaee8 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:39:59.504260 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-f5ce5ff0-5e93-49e5-a86a-90f62ffaaee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:59.509286 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ebffb0-c5f4-4222-b8ed-9ae25171b696 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:59.512493 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-62ebffb0-c5f4-4222-b8ed-9ae25171b696 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:59.568657 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1059/2132] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:39:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:39:59.577200 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea367f0b-1451-4773-b157-03f0507c67be None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:59.577509 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea367f0b-1451-4773-b157-03f0507c67be None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:59.577810 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea367f0b-1451-4773-b157-03f0507c67be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:59.587039 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ebffb0-c5f4-4222-b8ed-9ae25171b696 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:59.592068 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e503ddd0-1fb6-48b1-8851-3f2d27579bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:59.592720 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e503ddd0-1fb6-48b1-8851-3f2d27579bbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:59.593866 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e503ddd0-1fb6-48b1-8851-3f2d27579bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:59.593866 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e503ddd0-1fb6-48b1-8851-3f2d27579bbd admin admin] PATH_INFO: `/v3/users/47dab92a41c54285b4c1262ef5512f52` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:59.594573 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e503ddd0-1fb6-48b1-8851-3f2d27579bbd admin admin] RBAC: Authorizing `identity:delete_user(user_id=47dab92a41c54285b4c1262ef5512f52)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:59.596770 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e503ddd0-1fb6-48b1-8851-3f2d27579bbd admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:59.600410 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-ea367f0b-1451-4773-b157-03f0507c67be None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:39:59.601696 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-ea367f0b-1451-4773-b157-03f0507c67be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:39:59.612257 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1060/2133] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:39:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 07:39:59.627258 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea367f0b-1451-4773-b157-03f0507c67be None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:59.635246 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea367f0b-1451-4773-b157-03f0507c67be None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:59.641107 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da23c8ea-6b96-44af-87d8-831b79ad3fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:59.641919 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da23c8ea-6b96-44af-87d8-831b79ad3fe8 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:59.642413 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da23c8ea-6b96-44af-87d8-831b79ad3fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:59.642692 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da23c8ea-6b96-44af-87d8-831b79ad3fe8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:59.643382 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da23c8ea-6b96-44af-87d8-831b79ad3fe8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:39:59.643689 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da23c8ea-6b96-44af-87d8-831b79ad3fe8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:59.645368 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da23c8ea-6b96-44af-87d8-831b79ad3fe8 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:59.676596 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1061/2134] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 15 07:39:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:39:59.681338 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da23c8ea-6b96-44af-87d8-831b79ad3fe8 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:59.693908 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da23c8ea-6b96-44af-87d8-831b79ad3fe8 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:39:59.699606 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f49ee6-dd06-4477-99b3-213691107869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:39:59.700447 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f49ee6-dd06-4477-99b3-213691107869 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:59.700857 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f49ee6-dd06-4477-99b3-213691107869 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:59.701140 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f49ee6-dd06-4477-99b3-213691107869 admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:59.701878 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f49ee6-dd06-4477-99b3-213691107869 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:39:59.720237 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f49ee6-dd06-4477-99b3-213691107869 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:39:59.822825 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-e503ddd0-1fb6-48b1-8851-3f2d27579bbd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47dab92a41c54285b4c1262ef5512f52'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:59.872567 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1062/2135] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:39:59 2022] POST /identity/v3/projects => generated 367 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:39:59.876790 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f49ee6-dd06-4477-99b3-213691107869 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:39:59.900022 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-e503ddd0-1fb6-48b1-8851-3f2d27579bbd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47dab92a41c54285b4c1262ef5512f52'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:59.900704 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-e503ddd0-1fb6-48b1-8851-3f2d27579bbd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47dab92a41c54285b4c1262ef5512f52'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:59.901332 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-e503ddd0-1fb6-48b1-8851-3f2d27579bbd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47dab92a41c54285b4c1262ef5512f52'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:59.921983 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-e503ddd0-1fb6-48b1-8851-3f2d27579bbd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47dab92a41c54285b4c1262ef5512f52'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:39:59.958330 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:39:59.967618 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1074/2136] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:39:59 2022] DELETE /identity/v3/users/47dab92a41c54285b4c1262ef5512f52 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 07:39:59.971720 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a0afe4-9a35-450b-9057-62039eec5e54 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:39:59.972264 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a0afe4-9a35-450b-9057-62039eec5e54 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:39:59.972767 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a0afe4-9a35-450b-9057-62039eec5e54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:39:59.999203 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-91a0afe4-9a35-450b-9057-62039eec5e54 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:40:00.001332 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f49ee6-dd06-4477-99b3-213691107869 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:00.006573 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-91a0afe4-9a35-450b-9057-62039eec5e54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:00.086580 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d8a84b-e633-48a4-90d4-9cdcd7946472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:00.087578 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d8a84b-e633-48a4-90d4-9cdcd7946472 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:00.088587 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d8a84b-e633-48a4-90d4-9cdcd7946472 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:00.089024 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d8a84b-e633-48a4-90d4-9cdcd7946472 admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:00.090900 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d8a84b-e633-48a4-90d4-9cdcd7946472 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:00.092391 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d8a84b-e633-48a4-90d4-9cdcd7946472 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:00.122475 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1075/2137] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:39:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:40:00.132887 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a0afe4-9a35-450b-9057-62039eec5e54 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:00.136839 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-91a0afe4-9a35-450b-9057-62039eec5e54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:00.188119 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1063/2138] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:39:59 2022] POST /identity/v3/users => generated 284 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:40:00.192267 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d8a84b-e633-48a4-90d4-9cdcd7946472 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:00.194031 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-80d8a84b-e633-48a4-90d4-9cdcd7946472 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:00.212040 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d8a84b-e633-48a4-90d4-9cdcd7946472 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:00.228276 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a0afe4-9a35-450b-9057-62039eec5e54 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:00.231874 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86e3621-6663-4a85-bee6-b77feb5c1d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:00.232517 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86e3621-6663-4a85-bee6-b77feb5c1d1d admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:00.232827 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86e3621-6663-4a85-bee6-b77feb5c1d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:00.233084 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86e3621-6663-4a85-bee6-b77feb5c1d1d admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:00.233774 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86e3621-6663-4a85-bee6-b77feb5c1d1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:00.236138 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86e3621-6663-4a85-bee6-b77feb5c1d1d admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:00.237719 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b4e489-4fa7-42ba-9162-8334a97ba19f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:00.239150 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b4e489-4fa7-42ba-9162-8334a97ba19f service neutron] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:00.240215 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b4e489-4fa7-42ba-9162-8334a97ba19f service neutron] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:00.240672 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b4e489-4fa7-42ba-9162-8334a97ba19f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:00.250914 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1064/2139] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:40:00 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 07:40:00.252984 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86e3621-6663-4a85-bee6-b77feb5c1d1d admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:00.260344 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:40:00.263867 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b4e489-4fa7-42ba-9162-8334a97ba19f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:00.265386 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b4e489-4fa7-42ba-9162-8334a97ba19f service neutron] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:00.270922 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86e3621-6663-4a85-bee6-b77feb5c1d1d admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:00.278295 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583aaf87-beae-4e5a-bed1-6af736a6cc44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:00.278974 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583aaf87-beae-4e5a-bed1-6af736a6cc44 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:00.279260 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583aaf87-beae-4e5a-bed1-6af736a6cc44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:00.279540 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583aaf87-beae-4e5a-bed1-6af736a6cc44 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:00.280198 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583aaf87-beae-4e5a-bed1-6af736a6cc44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:00.284072 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583aaf87-beae-4e5a-bed1-6af736a6cc44 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:00.287138 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1076/2140] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:40:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 07:40:00.303498 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1065/2141] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:40:00 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 07:40:00.304693 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b4e489-4fa7-42ba-9162-8334a97ba19f service neutron] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:00.316226 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b4e489-4fa7-42ba-9162-8334a97ba19f service neutron] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:00.320885 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f872756c-5df3-405d-8e19-0ee85b27b19c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:00.321540 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f872756c-5df3-405d-8e19-0ee85b27b19c admin admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:00.321867 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f872756c-5df3-405d-8e19-0ee85b27b19c admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:00.322144 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f872756c-5df3-405d-8e19-0ee85b27b19c admin admin] PATH_INFO: `/v3/projects/1c989f6976354d3aab8fe4ee30501c12/users/048a8dccdce64e09bc87c65683ee34ef/roles/053e12af84e943e699460a9d76c2072b` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:00.338285 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f872756c-5df3-405d-8e19-0ee85b27b19c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c989f6976354d3aab8fe4ee30501c12, user_id=048a8dccdce64e09bc87c65683ee34ef, role_id=053e12af84e943e699460a9d76c2072b)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:00.341454 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f872756c-5df3-405d-8e19-0ee85b27b19c admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:00.438902 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1077/2142] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:40:00 2022] PUT /identity/v3/projects/1c989f6976354d3aab8fe4ee30501c12/users/048a8dccdce64e09bc87c65683ee34ef/roles/053e12af84e943e699460a9d76c2072b => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:40:00.450002 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecc3168-f718-4cde-81f2-933d00bac535 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:00.450002 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecc3168-f718-4cde-81f2-933d00bac535 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:00.450307 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecc3168-f718-4cde-81f2-933d00bac535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:00.482761 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-6ecc3168-f718-4cde-81f2-933d00bac535 None None] MFA Rules not processed for user `048a8dccdce64e09bc87c65683ee34ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:40:00.483984 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-6ecc3168-f718-4cde-81f2-933d00bac535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:00.497344 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d302ae95-1fcf-4345-b027-7e1ced7872ef None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:00.497671 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d302ae95-1fcf-4345-b027-7e1ced7872ef None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:00.498001 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d302ae95-1fcf-4345-b027-7e1ced7872ef None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:00.499532 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1078/2143] 10.222.0.42 () {54 vars in 921 bytes} [Thu Sep 15 07:40:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:40:00.556865 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1066/2144] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:40:00 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:40:00.562701 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d302ae95-1fcf-4345-b027-7e1ced7872ef None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:00.632177 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d302ae95-1fcf-4345-b027-7e1ced7872ef None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:00.638023 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3bd46e-e0d9-4380-ac13-bc017e1a6a95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:00.638713 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3bd46e-e0d9-4380-ac13-bc017e1a6a95 admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:00.638998 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3bd46e-e0d9-4380-ac13-bc017e1a6a95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:00.639281 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3bd46e-e0d9-4380-ac13-bc017e1a6a95 admin admin] PATH_INFO: `/v3/projects` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:00.639944 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3bd46e-e0d9-4380-ac13-bc017e1a6a95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:00.641925 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3bd46e-e0d9-4380-ac13-bc017e1a6a95 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:00.715714 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad952229-630c-4c3f-ac2e-0ec9f5b6d077 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:00.715913 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad952229-630c-4c3f-ac2e-0ec9f5b6d077 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:00.716184 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad952229-630c-4c3f-ac2e-0ec9f5b6d077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:00.730980 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1079/2145] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:40:00 2022] POST /identity/v3/projects => generated 367 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 07:40:00.734636 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3bd46e-e0d9-4380-ac13-bc017e1a6a95 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:00.737381 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-ad952229-630c-4c3f-ac2e-0ec9f5b6d077 None None] MFA Rules not processed for user `dd87846b9a134248b28e0521b0a880c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:40:00.738631 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-ad952229-630c-4c3f-ac2e-0ec9f5b6d077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:00.787690 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:40:00.812194 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3bd46e-e0d9-4380-ac13-bc017e1a6a95 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:00.812667 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1067/2146] 10.222.0.42 () {58 vars in 1018 bytes} [Thu Sep 15 07:40:00 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 07:40:00.817279 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3baaf6-5530-4cd4-8000-2618fe9c62fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:00.817951 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3baaf6-5530-4cd4-8000-2618fe9c62fe admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:00.818230 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3baaf6-5530-4cd4-8000-2618fe9c62fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:00.818502 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3baaf6-5530-4cd4-8000-2618fe9c62fe admin admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:00.819124 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3baaf6-5530-4cd4-8000-2618fe9c62fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:00.821055 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3baaf6-5530-4cd4-8000-2618fe9c62fe admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:00.823284 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad952229-630c-4c3f-ac2e-0ec9f5b6d077 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:00.825338 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-ad952229-630c-4c3f-ac2e-0ec9f5b6d077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:00.837155 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:40:00.838035 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad952229-630c-4c3f-ac2e-0ec9f5b6d077 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:00.843265 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f6108e-24d4-4098-bac3-5ade79bdf2f2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd87846b9a134248b28e0521b0a880c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:00.846911 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f6108e-24d4-4098-bac3-5ade79bdf2f2 service placement] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:00.846911 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f6108e-24d4-4098-bac3-5ade79bdf2f2 service placement] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:00.846911 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f6108e-24d4-4098-bac3-5ade79bdf2f2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:00.852650 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f6108e-24d4-4098-bac3-5ade79bdf2f2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:00.854253 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f6108e-24d4-4098-bac3-5ade79bdf2f2 service placement] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:00.864047 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1068/2147] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:40:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 07:40:00.900529 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1080/2148] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:40:00 2022] POST /identity/v3/users => generated 290 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 07:40:00.906070 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f6108e-24d4-4098-bac3-5ade79bdf2f2 service placement] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:00.914122 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3baaf6-5530-4cd4-8000-2618fe9c62fe admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:00.918031 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-1d3baaf6-5530-4cd4-8000-2618fe9c62fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:00.918789 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f6108e-24d4-4098-bac3-5ade79bdf2f2 service placement] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:00.924293 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2fd662-6a8c-41b8-809c-0bf96c82499b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:00.924920 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2fd662-6a8c-41b8-809c-0bf96c82499b admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:00.925201 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2fd662-6a8c-41b8-809c-0bf96c82499b admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:00.925831 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2fd662-6a8c-41b8-809c-0bf96c82499b admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:00.926296 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2fd662-6a8c-41b8-809c-0bf96c82499b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:00.928155 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2fd662-6a8c-41b8-809c-0bf96c82499b admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:00.936866 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3baaf6-5530-4cd4-8000-2618fe9c62fe admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:00.949469 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1069/2149] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:40:00 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:40:00.949653 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed91c65-42a8-4636-a399-b612d96e682b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:00.949653 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed91c65-42a8-4636-a399-b612d96e682b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:00.949653 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed91c65-42a8-4636-a399-b612d96e682b admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:00.949653 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed91c65-42a8-4636-a399-b612d96e682b admin admin] PATH_INFO: `/v3/users/1d51e6d2afe1436e8a65b895c83bdd38` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:00.950290 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2fd662-6a8c-41b8-809c-0bf96c82499b admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:00.950949 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed91c65-42a8-4636-a399-b612d96e682b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d51e6d2afe1436e8a65b895c83bdd38)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:00.954666 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed91c65-42a8-4636-a399-b612d96e682b admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:00.962869 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2fd662-6a8c-41b8-809c-0bf96c82499b admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:00.968782 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3097eab6-a9c8-42c3-b8ca-903f87e64291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:00.969803 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3097eab6-a9c8-42c3-b8ca-903f87e64291 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:00.969935 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3097eab6-a9c8-42c3-b8ca-903f87e64291 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:00.970137 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3097eab6-a9c8-42c3-b8ca-903f87e64291 admin admin] PATH_INFO: `/v3/projects/5005fe9692f5404c8f9482df9dc6a920/users/25474b1626db4592b948fb17977d3479/roles/6bbb643e0d8d4bc0a9b27c26edf59216` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:00.986528 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3097eab6-a9c8-42c3-b8ca-903f87e64291 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5005fe9692f5404c8f9482df9dc6a920, user_id=25474b1626db4592b948fb17977d3479, role_id=6bbb643e0d8d4bc0a9b27c26edf59216)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:00.989192 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3097eab6-a9c8-42c3-b8ca-903f87e64291 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:01.080411 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1070/2150] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:40:00 2022] PUT /identity/v3/projects/5005fe9692f5404c8f9482df9dc6a920/users/25474b1626db4592b948fb17977d3479/roles/6bbb643e0d8d4bc0a9b27c26edf59216 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 07:40:01.084209 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3097eab6-a9c8-42c3-b8ca-903f87e64291 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:01.147570 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3097eab6-a9c8-42c3-b8ca-903f87e64291 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:01.151562 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca87a18-875f-4f4c-9996-1ba102b1a30d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:01.153033 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca87a18-875f-4f4c-9996-1ba102b1a30d admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:01.153320 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca87a18-875f-4f4c-9996-1ba102b1a30d admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:01.153612 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca87a18-875f-4f4c-9996-1ba102b1a30d admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:01.154341 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca87a18-875f-4f4c-9996-1ba102b1a30d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:01.156548 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca87a18-875f-4f4c-9996-1ba102b1a30d admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:01.170422 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1071/2151] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:40:01 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 07:40:01.170422 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca87a18-875f-4f4c-9996-1ba102b1a30d admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:01.177562 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-3ca87a18-875f-4f4c-9996-1ba102b1a30d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:01.180977 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-0ed91c65-42a8-4636-a399-b612d96e682b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d51e6d2afe1436e8a65b895c83bdd38'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:01.192827 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca87a18-875f-4f4c-9996-1ba102b1a30d admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:01.197548 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c057d033-bd7b-47c8-9fd2-3e5ea09ab58a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:01.198205 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c057d033-bd7b-47c8-9fd2-3e5ea09ab58a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:01.198487 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c057d033-bd7b-47c8-9fd2-3e5ea09ab58a admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:01.199403 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c057d033-bd7b-47c8-9fd2-3e5ea09ab58a admin admin] PATH_INFO: `/v3/projects/689f09db29af457098dbffb8b346ee27` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:01.200832 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c057d033-bd7b-47c8-9fd2-3e5ea09ab58a admin admin] RBAC: Authorizing `identity:delete_project(project_id=689f09db29af457098dbffb8b346ee27)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:01.202768 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c057d033-bd7b-47c8-9fd2-3e5ea09ab58a admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:01.265321 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-0ed91c65-42a8-4636-a399-b612d96e682b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d51e6d2afe1436e8a65b895c83bdd38'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:01.265925 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-0ed91c65-42a8-4636-a399-b612d96e682b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d51e6d2afe1436e8a65b895c83bdd38'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:01.266534 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-0ed91c65-42a8-4636-a399-b612d96e682b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d51e6d2afe1436e8a65b895c83bdd38'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:01.274006 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-0ed91c65-42a8-4636-a399-b612d96e682b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d51e6d2afe1436e8a65b895c83bdd38'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:01.296395 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1081/2152] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:40:00 2022] DELETE /identity/v3/users/1d51e6d2afe1436e8a65b895c83bdd38 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 07:40:01.297270 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed91c65-42a8-4636-a399-b612d96e682b admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:01.298666 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-0ed91c65-42a8-4636-a399-b612d96e682b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:01.316111 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-c057d033-bd7b-47c8-9fd2-3e5ea09ab58a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '689f09db29af457098dbffb8b346ee27', 'request_id': 'req-c057d033-bd7b-47c8-9fd2-3e5ea09ab58a'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:01.322918 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-c057d033-bd7b-47c8-9fd2-3e5ea09ab58a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '689f09db29af457098dbffb8b346ee27', 'request_id': 'req-c057d033-bd7b-47c8-9fd2-3e5ea09ab58a'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:01.370139 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed91c65-42a8-4636-a399-b612d96e682b admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:01.391519 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-c057d033-bd7b-47c8-9fd2-3e5ea09ab58a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '689f09db29af457098dbffb8b346ee27', 'request_id': 'req-c057d033-bd7b-47c8-9fd2-3e5ea09ab58a'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:01.405282 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-c057d033-bd7b-47c8-9fd2-3e5ea09ab58a admin admin] The token cache is being invalidate because project 689f09db29af457098dbffb8b346ee27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:40:01.405723 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-c057d033-bd7b-47c8-9fd2-3e5ea09ab58a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:01.406494 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-c057d033-bd7b-47c8-9fd2-3e5ea09ab58a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:01.408051 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1072/2153] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:40:01 2022] DELETE /identity/v3/projects/689f09db29af457098dbffb8b346ee27 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:40:01.412037 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c057d033-bd7b-47c8-9fd2-3e5ea09ab58a admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:01.413438 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-c057d033-bd7b-47c8-9fd2-3e5ea09ab58a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:01.416822 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:40:01.428254 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:40:01.439743 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e96241b-35b2-4bb1-bf87-b020075cc429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:01.440422 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e96241b-35b2-4bb1-bf87-b020075cc429 admin admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:01.440700 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e96241b-35b2-4bb1-bf87-b020075cc429 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:01.440975 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e96241b-35b2-4bb1-bf87-b020075cc429 admin admin] PATH_INFO: `/v3/projects/5005fe9692f5404c8f9482df9dc6a920/users/25474b1626db4592b948fb17977d3479/roles/6bbb643e0d8d4bc0a9b27c26edf59216` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:01.442898 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e96241b-35b2-4bb1-bf87-b020075cc429 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5005fe9692f5404c8f9482df9dc6a920, user_id=25474b1626db4592b948fb17977d3479, role_id=6bbb643e0d8d4bc0a9b27c26edf59216)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:01.445877 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e96241b-35b2-4bb1-bf87-b020075cc429 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:01.457753 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1082/2154] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:40:01 2022] PUT /identity/v3/projects/5005fe9692f5404c8f9482df9dc6a920/users/25474b1626db4592b948fb17977d3479/roles/6bbb643e0d8d4bc0a9b27c26edf59216 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:40:01.469541 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2921e275-1b40-4e74-a018-1b10d21a7193 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:01.469855 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2921e275-1b40-4e74-a018-1b10d21a7193 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:01.470140 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2921e275-1b40-4e74-a018-1b10d21a7193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:01.497162 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c057d033-bd7b-47c8-9fd2-3e5ea09ab58a admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:01.500457 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-2921e275-1b40-4e74-a018-1b10d21a7193 None None] MFA Rules not processed for user `25474b1626db4592b948fb17977d3479`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:40:01.501742 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-2921e275-1b40-4e74-a018-1b10d21a7193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:01.562627 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b1d01d-510a-4643-b34b-275a0eebf162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:01.566061 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b1d01d-510a-4643-b34b-275a0eebf162 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:01.566061 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b1d01d-510a-4643-b34b-275a0eebf162 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:01.566061 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b1d01d-510a-4643-b34b-275a0eebf162 admin admin] PATH_INFO: `/v3/users/f16a3657eb4e46aea71b4b06b158db49` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:01.567629 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b1d01d-510a-4643-b34b-275a0eebf162 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f16a3657eb4e46aea71b4b06b158db49)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:01.570643 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b1d01d-510a-4643-b34b-275a0eebf162 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:01.581872 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1083/2155] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:40:01 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 07:40:01.582764 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2921e275-1b40-4e74-a018-1b10d21a7193 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:01.584344 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-2921e275-1b40-4e74-a018-1b10d21a7193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:01.600644 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2921e275-1b40-4e74-a018-1b10d21a7193 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:01.606771 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ef5439-31fc-4693-a7cd-3df302f572eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:01.607415 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ef5439-31fc-4693-a7cd-3df302f572eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:01.607699 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ef5439-31fc-4693-a7cd-3df302f572eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:01.607976 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ef5439-31fc-4693-a7cd-3df302f572eb admin admin] PATH_INFO: `/v3/projects/9836d4c25e0e46a8a6245bb7b74c55ce` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:01.609115 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ef5439-31fc-4693-a7cd-3df302f572eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=9836d4c25e0e46a8a6245bb7b74c55ce)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:01.611152 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ef5439-31fc-4693-a7cd-3df302f572eb admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:01.794564 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-d0ef5439-31fc-4693-a7cd-3df302f572eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9836d4c25e0e46a8a6245bb7b74c55ce', 'request_id': 'req-d0ef5439-31fc-4693-a7cd-3df302f572eb'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:01.794928 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-a7b1d01d-510a-4643-b34b-275a0eebf162 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f16a3657eb4e46aea71b4b06b158db49'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:01.795784 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-a7b1d01d-510a-4643-b34b-275a0eebf162 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f16a3657eb4e46aea71b4b06b158db49'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:01.802167 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-d0ef5439-31fc-4693-a7cd-3df302f572eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9836d4c25e0e46a8a6245bb7b74c55ce', 'request_id': 'req-d0ef5439-31fc-4693-a7cd-3df302f572eb'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:01.871597 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-a7b1d01d-510a-4643-b34b-275a0eebf162 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f16a3657eb4e46aea71b4b06b158db49'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:01.872271 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-a7b1d01d-510a-4643-b34b-275a0eebf162 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f16a3657eb4e46aea71b4b06b158db49'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:01.879831 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-a7b1d01d-510a-4643-b34b-275a0eebf162 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f16a3657eb4e46aea71b4b06b158db49'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:01.897809 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1073/2156] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:40:01 2022] DELETE /identity/v3/users/f16a3657eb4e46aea71b4b06b158db49 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:40:01.898599 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b1d01d-510a-4643-b34b-275a0eebf162 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:01.900047 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-a7b1d01d-510a-4643-b34b-275a0eebf162 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:01.928094 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-d0ef5439-31fc-4693-a7cd-3df302f572eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9836d4c25e0e46a8a6245bb7b74c55ce', 'request_id': 'req-d0ef5439-31fc-4693-a7cd-3df302f572eb'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:01.941561 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-d0ef5439-31fc-4693-a7cd-3df302f572eb admin admin] The token cache is being invalidate because project 9836d4c25e0e46a8a6245bb7b74c55ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:40:01.941997 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-d0ef5439-31fc-4693-a7cd-3df302f572eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:01.942748 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-d0ef5439-31fc-4693-a7cd-3df302f572eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:01.948763 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1084/2157] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:40:01 2022] DELETE /identity/v3/projects/9836d4c25e0e46a8a6245bb7b74c55ce => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:40:01.949735 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c3e2b-0270-4eb5-8871-e4d21c0e245b None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:01.949974 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c3e2b-0270-4eb5-8871-e4d21c0e245b None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:01.950375 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c3e2b-0270-4eb5-8871-e4d21c0e245b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:01.970665 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-a57c3e2b-0270-4eb5-8871-e4d21c0e245b None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:40:01.971723 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b1d01d-510a-4643-b34b-275a0eebf162 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:01.971869 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-a57c3e2b-0270-4eb5-8871-e4d21c0e245b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:01.975923 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdaf975-49aa-4f88-87f8-3182579dcd76 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecf6642b7b84d3c9756bb41ab3349d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:01.976778 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdaf975-49aa-4f88-87f8-3182579dcd76 service cinder] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:01.977119 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdaf975-49aa-4f88-87f8-3182579dcd76 service cinder] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:01.977948 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdaf975-49aa-4f88-87f8-3182579dcd76 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:01.980777 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-0bdaf975-49aa-4f88-87f8-3182579dcd76 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:02.019442 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:40:02.038294 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1085/2158] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:40:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 07:40:02.046532 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f2fde5-86fc-4651-8f3f-9cd17b7daba1 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:02.046751 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f2fde5-86fc-4651-8f3f-9cd17b7daba1 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:02.047036 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f2fde5-86fc-4651-8f3f-9cd17b7daba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:02.053325 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdaf975-49aa-4f88-87f8-3182579dcd76 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:02.054859 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdaf975-49aa-4f88-87f8-3182579dcd76 service cinder] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:02.064534 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1074/2159] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:40:01 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:40:02.068593 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-50f2fde5-86fc-4651-8f3f-9cd17b7daba1 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:40:02.068593 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-50f2fde5-86fc-4651-8f3f-9cd17b7daba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:02.076345 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1086/2160] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:40:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 07:40:02.080694 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdaf975-49aa-4f88-87f8-3182579dcd76 service cinder] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:02.092215 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdaf975-49aa-4f88-87f8-3182579dcd76 service cinder] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:02.098373 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:40:02.099921 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f6d935-f21a-4e19-bc80-08cec25ef6c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:02.100059 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f6d935-f21a-4e19-bc80-08cec25ef6c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:02.100322 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f6d935-f21a-4e19-bc80-08cec25ef6c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:02.100566 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f6d935-f21a-4e19-bc80-08cec25ef6c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:02.101236 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f6d935-f21a-4e19-bc80-08cec25ef6c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:40:02.101539 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f6d935-f21a-4e19-bc80-08cec25ef6c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:02.103145 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f6d935-f21a-4e19-bc80-08cec25ef6c3 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:02.115274 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1075/2161] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 15 07:40:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:40:02.119406 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f2fde5-86fc-4651-8f3f-9cd17b7daba1 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:02.128895 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f2fde5-86fc-4651-8f3f-9cd17b7daba1 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:02.135749 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a65f3d-eead-4b51-91a2-72a76542ab51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:02.136907 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a65f3d-eead-4b51-91a2-72a76542ab51 admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:02.137192 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a65f3d-eead-4b51-91a2-72a76542ab51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:02.137745 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a65f3d-eead-4b51-91a2-72a76542ab51 admin admin] PATH_INFO: `/v3/projects` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:02.138149 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a65f3d-eead-4b51-91a2-72a76542ab51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:02.140069 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a65f3d-eead-4b51-91a2-72a76542ab51 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:02.247401 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1087/2162] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:40:02 2022] POST /identity/v3/projects => generated 391 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:40:02.248605 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f6d935-f21a-4e19-bc80-08cec25ef6c3 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:02.258725 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f6d935-f21a-4e19-bc80-08cec25ef6c3 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:02.282926 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a65f3d-eead-4b51-91a2-72a76542ab51 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:02.284741 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-91a65f3d-eead-4b51-91a2-72a76542ab51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:02.317361 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabbead0-f4c3-4514-b79c-428908c0889d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:02.317917 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabbead0-f4c3-4514-b79c-428908c0889d admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:02.318202 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabbead0-f4c3-4514-b79c-428908c0889d admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:02.318466 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabbead0-f4c3-4514-b79c-428908c0889d admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:02.319141 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabbead0-f4c3-4514-b79c-428908c0889d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:02.321038 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabbead0-f4c3-4514-b79c-428908c0889d admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:02.342569 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a65f3d-eead-4b51-91a2-72a76542ab51 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:02.346512 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25739555-e730-44dc-8698-6311c3948ff2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecf6642b7b84d3c9756bb41ab3349d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:02.347127 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25739555-e730-44dc-8698-6311c3948ff2 service cinder] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:02.347404 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25739555-e730-44dc-8698-6311c3948ff2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:02.347716 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25739555-e730-44dc-8698-6311c3948ff2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:02.349549 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-25739555-e730-44dc-8698-6311c3948ff2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:02.411855 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25739555-e730-44dc-8698-6311c3948ff2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:02.413296 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25739555-e730-44dc-8698-6311c3948ff2 service cinder] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:02.422390 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1088/2163] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:40:02 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 07:40:02.423278 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25739555-e730-44dc-8698-6311c3948ff2 service cinder] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:02.424581 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-25739555-e730-44dc-8698-6311c3948ff2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:02.441307 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1076/2164] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:40:02 2022] POST /identity/v3/users => generated 296 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:40:02.441715 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25739555-e730-44dc-8698-6311c3948ff2 service cinder] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:02.442556 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabbead0-f4c3-4514-b79c-428908c0889d admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:02.444244 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfcdd7b-5832-4242-a3fc-785f4007d6cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:02.444883 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfcdd7b-5832-4242-a3fc-785f4007d6cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:02.445165 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfcdd7b-5832-4242-a3fc-785f4007d6cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:02.445441 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfcdd7b-5832-4242-a3fc-785f4007d6cc admin admin] PATH_INFO: `/v3/projects/a3dedcc6d611402db8e9acb55be14a04` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:02.446658 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfcdd7b-5832-4242-a3fc-785f4007d6cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3dedcc6d611402db8e9acb55be14a04)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:02.448587 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfcdd7b-5832-4242-a3fc-785f4007d6cc admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:02.453495 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabbead0-f4c3-4514-b79c-428908c0889d admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:02.458481 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0223e8-c0ea-4cab-8c06-abf8766bdfba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:02.459006 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0223e8-c0ea-4cab-8c06-abf8766bdfba admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:02.459287 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0223e8-c0ea-4cab-8c06-abf8766bdfba admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:02.459566 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0223e8-c0ea-4cab-8c06-abf8766bdfba admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:02.460204 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0223e8-c0ea-4cab-8c06-abf8766bdfba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:02.461877 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0223e8-c0ea-4cab-8c06-abf8766bdfba admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:02.473870 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1077/2165] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:40:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:40:02.477847 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0223e8-c0ea-4cab-8c06-abf8766bdfba admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:02.489777 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0223e8-c0ea-4cab-8c06-abf8766bdfba admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:02.494562 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ff0774-b530-4b19-ba65-42e4aec4f9cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:02.495158 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ff0774-b530-4b19-ba65-42e4aec4f9cb admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:02.495464 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ff0774-b530-4b19-ba65-42e4aec4f9cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:02.496284 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ff0774-b530-4b19-ba65-42e4aec4f9cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:02.497247 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ff0774-b530-4b19-ba65-42e4aec4f9cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:02.498864 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ff0774-b530-4b19-ba65-42e4aec4f9cb admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:02.509385 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1078/2166] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:40:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:40:02.514566 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ff0774-b530-4b19-ba65-42e4aec4f9cb admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:02.524355 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ff0774-b530-4b19-ba65-42e4aec4f9cb admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:02.529309 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a61c8a-d235-4140-b614-3c0f9cf457a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:02.529984 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a61c8a-d235-4140-b614-3c0f9cf457a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:02.530265 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a61c8a-d235-4140-b614-3c0f9cf457a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:02.530541 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a61c8a-d235-4140-b614-3c0f9cf457a1 admin admin] PATH_INFO: `/v3/projects/cdc69db806d24d1f8eb18aadaaafabcb/users/21de3a85b2df4af2833a24ab02f477c8/roles/053e12af84e943e699460a9d76c2072b` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:02.546401 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a61c8a-d235-4140-b614-3c0f9cf457a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdc69db806d24d1f8eb18aadaaafabcb, user_id=21de3a85b2df4af2833a24ab02f477c8, role_id=053e12af84e943e699460a9d76c2072b)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:02.549150 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a61c8a-d235-4140-b614-3c0f9cf457a1 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:02.637333 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1079/2167] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:40:02 2022] PUT /identity/v3/projects/cdc69db806d24d1f8eb18aadaaafabcb/users/21de3a85b2df4af2833a24ab02f477c8/roles/053e12af84e943e699460a9d76c2072b => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 07:40:02.642307 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e09c8e5-6bc7-4a4b-a779-f6b6326c810e None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:02.642601 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e09c8e5-6bc7-4a4b-a779-f6b6326c810e None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:02.642884 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e09c8e5-6bc7-4a4b-a779-f6b6326c810e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:02.646571 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-6dfcdd7b-5832-4242-a3fc-785f4007d6cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3dedcc6d611402db8e9acb55be14a04', 'request_id': 'req-6dfcdd7b-5832-4242-a3fc-785f4007d6cc'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:02.657122 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-6dfcdd7b-5832-4242-a3fc-785f4007d6cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3dedcc6d611402db8e9acb55be14a04', 'request_id': 'req-6dfcdd7b-5832-4242-a3fc-785f4007d6cc'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:02.666216 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-4e09c8e5-6bc7-4a4b-a779-f6b6326c810e None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:40:02.668108 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-4e09c8e5-6bc7-4a4b-a779-f6b6326c810e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:02.729285 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1080/2168] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:40:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 07:40:02.739473 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e09c8e5-6bc7-4a4b-a779-f6b6326c810e None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:02.739816 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-6dfcdd7b-5832-4242-a3fc-785f4007d6cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3dedcc6d611402db8e9acb55be14a04', 'request_id': 'req-6dfcdd7b-5832-4242-a3fc-785f4007d6cc'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:02.741629 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-4e09c8e5-6bc7-4a4b-a779-f6b6326c810e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:02.744516 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:40:02.754415 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-6dfcdd7b-5832-4242-a3fc-785f4007d6cc admin admin] The token cache is being invalidate because project a3dedcc6d611402db8e9acb55be14a04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:40:02.754724 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-6dfcdd7b-5832-4242-a3fc-785f4007d6cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:02.755395 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-6dfcdd7b-5832-4242-a3fc-785f4007d6cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:02.757011 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1089/2169] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:40:02 2022] DELETE /identity/v3/projects/a3dedcc6d611402db8e9acb55be14a04 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:40:02.762497 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1205e312-cc27-4679-a976-58834cc85e0b None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:02.762849 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1205e312-cc27-4679-a976-58834cc85e0b None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:02.763179 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1205e312-cc27-4679-a976-58834cc85e0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:02.783575 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-1205e312-cc27-4679-a976-58834cc85e0b None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:40:02.784811 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-1205e312-cc27-4679-a976-58834cc85e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:02.793697 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1090/2170] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:40:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:40:02.801304 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e09c8e5-6bc7-4a4b-a779-f6b6326c810e None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:02.801439 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4388f462-b238-45b6-84a3-55ae50fb8160 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:02.801647 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4388f462-b238-45b6-84a3-55ae50fb8160 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:02.801928 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4388f462-b238-45b6-84a3-55ae50fb8160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:02.805168 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c631a975-71a5-465e-bfb9-6b2cb9a9e3fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:02.805808 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c631a975-71a5-465e-bfb9-6b2cb9a9e3fc service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:02.806093 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c631a975-71a5-465e-bfb9-6b2cb9a9e3fc service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:02.810656 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c631a975-71a5-465e-bfb9-6b2cb9a9e3fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:02.810656 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-c631a975-71a5-465e-bfb9-6b2cb9a9e3fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:02.821452 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c631a975-71a5-465e-bfb9-6b2cb9a9e3fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:02.822139 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-4388f462-b238-45b6-84a3-55ae50fb8160 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:40:02.823105 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c631a975-71a5-465e-bfb9-6b2cb9a9e3fc service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:02.823316 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-4388f462-b238-45b6-84a3-55ae50fb8160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:02.835599 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1091/2171] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:40:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:40:02.835840 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1081/2172] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:40:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:40:02.837187 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4388f462-b238-45b6-84a3-55ae50fb8160 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:02.840227 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:40:02.847563 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4388f462-b238-45b6-84a3-55ae50fb8160 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:02.853344 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f28d2c-0737-44d0-9e7b-7eabf307def5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:02.854019 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f28d2c-0737-44d0-9e7b-7eabf307def5 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:02.854296 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f28d2c-0737-44d0-9e7b-7eabf307def5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:02.854570 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f28d2c-0737-44d0-9e7b-7eabf307def5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:02.855751 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f28d2c-0737-44d0-9e7b-7eabf307def5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:40:02.856055 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f28d2c-0737-44d0-9e7b-7eabf307def5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:02.857684 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f28d2c-0737-44d0-9e7b-7eabf307def5 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:02.869409 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1092/2173] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 15 07:40:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:40:02.874512 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c631a975-71a5-465e-bfb9-6b2cb9a9e3fc service neutron] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:02.883924 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c631a975-71a5-465e-bfb9-6b2cb9a9e3fc service neutron] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:02.889052 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a5035c-4796-498d-a042-6d480f745452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:02.889758 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a5035c-4796-498d-a042-6d480f745452 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:02.890047 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a5035c-4796-498d-a042-6d480f745452 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:02.890327 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a5035c-4796-498d-a042-6d480f745452 admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:02.890963 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a5035c-4796-498d-a042-6d480f745452 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:02.893033 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a5035c-4796-498d-a042-6d480f745452 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:02.993454 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1082/2174] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:40:02 2022] POST /identity/v3/projects => generated 383 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:40:02.994407 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f28d2c-0737-44d0-9e7b-7eabf307def5 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:03.004458 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f28d2c-0737-44d0-9e7b-7eabf307def5 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:03.068318 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6428c794-7813-4536-a836-4688baa6d8a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:03.068937 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6428c794-7813-4536-a836-4688baa6d8a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:03.069219 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6428c794-7813-4536-a836-4688baa6d8a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:03.069512 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6428c794-7813-4536-a836-4688baa6d8a8 admin admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:03.070179 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6428c794-7813-4536-a836-4688baa6d8a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:03.072092 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6428c794-7813-4536-a836-4688baa6d8a8 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:03.163389 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1093/2175] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:40:02 2022] POST /identity/v3/users => generated 292 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:40:03.166733 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a5035c-4796-498d-a042-6d480f745452 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:03.180169 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a5035c-4796-498d-a042-6d480f745452 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:03.192247 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7f4026-3406-409b-bf0d-a140015764b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:03.192247 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7f4026-3406-409b-bf0d-a140015764b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:03.192247 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7f4026-3406-409b-bf0d-a140015764b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:03.192247 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7f4026-3406-409b-bf0d-a140015764b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:03.192822 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7f4026-3406-409b-bf0d-a140015764b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:03.193646 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7f4026-3406-409b-bf0d-a140015764b0 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:03.204801 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1083/2176] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:40:03 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 07:40:03.208567 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6428c794-7813-4536-a836-4688baa6d8a8 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:03.221163 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6428c794-7813-4536-a836-4688baa6d8a8 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:03.226101 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86630d50-0045-44e8-99d9-2c04e3d5a346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:03.226720 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86630d50-0045-44e8-99d9-2c04e3d5a346 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:03.227000 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86630d50-0045-44e8-99d9-2c04e3d5a346 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:03.227277 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86630d50-0045-44e8-99d9-2c04e3d5a346 admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:03.227922 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86630d50-0045-44e8-99d9-2c04e3d5a346 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:03.229517 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86630d50-0045-44e8-99d9-2c04e3d5a346 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:03.241499 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1094/2177] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:40:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:40:03.245803 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7f4026-3406-409b-bf0d-a140015764b0 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:03.255876 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7f4026-3406-409b-bf0d-a140015764b0 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:03.260710 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2183ee-b3eb-4b84-9f3d-6407ec393f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:03.261542 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2183ee-b3eb-4b84-9f3d-6407ec393f35 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:03.261877 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2183ee-b3eb-4b84-9f3d-6407ec393f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:03.262258 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2183ee-b3eb-4b84-9f3d-6407ec393f35 admin admin] PATH_INFO: `/v3/projects/69a24f9ec9544eb788b0d756bb47ffb5/users/fe558c6ceb674ed6b0b0ca5b97ef9cc7/roles/053e12af84e943e699460a9d76c2072b` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:03.277603 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2183ee-b3eb-4b84-9f3d-6407ec393f35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69a24f9ec9544eb788b0d756bb47ffb5, user_id=fe558c6ceb674ed6b0b0ca5b97ef9cc7, role_id=053e12af84e943e699460a9d76c2072b)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:03.280431 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2183ee-b3eb-4b84-9f3d-6407ec393f35 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:03.381341 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1084/2178] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:40:03 2022] PUT /identity/v3/projects/69a24f9ec9544eb788b0d756bb47ffb5/users/fe558c6ceb674ed6b0b0ca5b97ef9cc7/roles/053e12af84e943e699460a9d76c2072b => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:40:03.393518 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6787433-78fe-4e8e-b5e9-79ff58896a08 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:03.394009 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6787433-78fe-4e8e-b5e9-79ff58896a08 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:03.394109 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6787433-78fe-4e8e-b5e9-79ff58896a08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:03.423002 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-d6787433-78fe-4e8e-b5e9-79ff58896a08 None None] MFA Rules not processed for user `fe558c6ceb674ed6b0b0ca5b97ef9cc7`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:40:03.424206 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-d6787433-78fe-4e8e-b5e9-79ff58896a08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:03.499659 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1095/2179] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:40:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:40:03.510816 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2183ee-b3eb-4b84-9f3d-6407ec393f35 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:03.512639 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-6d2183ee-b3eb-4b84-9f3d-6407ec393f35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:03.570378 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2183ee-b3eb-4b84-9f3d-6407ec393f35 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:03.574685 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6b857a-9e06-4df7-8dfb-8dc761f77899 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecf6642b7b84d3c9756bb41ab3349d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:03.575305 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b857a-9e06-4df7-8dfb-8dc761f77899 service cinder] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:03.575594 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b857a-9e06-4df7-8dfb-8dc761f77899 service cinder] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:03.575870 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b857a-9e06-4df7-8dfb-8dc761f77899 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:03.584589 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6b857a-9e06-4df7-8dfb-8dc761f77899 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:03.586054 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6b857a-9e06-4df7-8dfb-8dc761f77899 service cinder] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:03.595477 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1085/2180] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:40:03 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:40:03.615436 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6787433-78fe-4e8e-b5e9-79ff58896a08 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:03.666351 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:40:03.688359 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6787433-78fe-4e8e-b5e9-79ff58896a08 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:03.693285 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35fb3b9-7896-4a5e-91cf-25579227780a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:03.693992 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35fb3b9-7896-4a5e-91cf-25579227780a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:03.694271 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35fb3b9-7896-4a5e-91cf-25579227780a admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:03.694545 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35fb3b9-7896-4a5e-91cf-25579227780a admin admin] PATH_INFO: `/v3/users/fe558c6ceb674ed6b0b0ca5b97ef9cc7` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:03.695703 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35fb3b9-7896-4a5e-91cf-25579227780a admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe558c6ceb674ed6b0b0ca5b97ef9cc7)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:03.697799 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35fb3b9-7896-4a5e-91cf-25579227780a admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:03.881812 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-c35fb3b9-7896-4a5e-91cf-25579227780a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe558c6ceb674ed6b0b0ca5b97ef9cc7'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:03.940192 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-c35fb3b9-7896-4a5e-91cf-25579227780a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe558c6ceb674ed6b0b0ca5b97ef9cc7'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:03.940926 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-c35fb3b9-7896-4a5e-91cf-25579227780a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe558c6ceb674ed6b0b0ca5b97ef9cc7'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:03.941614 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-c35fb3b9-7896-4a5e-91cf-25579227780a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe558c6ceb674ed6b0b0ca5b97ef9cc7'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:03.950433 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-c35fb3b9-7896-4a5e-91cf-25579227780a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe558c6ceb674ed6b0b0ca5b97ef9cc7'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:03.968406 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1096/2181] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:40:03 2022] DELETE /identity/v3/users/fe558c6ceb674ed6b0b0ca5b97ef9cc7 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:40:03.972899 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194e40de-0ae7-4400-b0b5-7c38ef99c42e None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:03.973219 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194e40de-0ae7-4400-b0b5-7c38ef99c42e None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:03.973552 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194e40de-0ae7-4400-b0b5-7c38ef99c42e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:03.986611 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:40:03.992736 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-194e40de-0ae7-4400-b0b5-7c38ef99c42e None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:40:03.993957 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-194e40de-0ae7-4400-b0b5-7c38ef99c42e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:04.053520 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1086/2182] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:40:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 07:40:04.064214 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35fb3b9-7896-4a5e-91cf-25579227780a admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:04.065746 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-c35fb3b9-7896-4a5e-91cf-25579227780a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:04.131641 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35fb3b9-7896-4a5e-91cf-25579227780a admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:04.147424 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1c97b8-bfdf-4b81-a666-03e5abc1685b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:04.148136 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1c97b8-bfdf-4b81-a666-03e5abc1685b service neutron] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:04.148424 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1c97b8-bfdf-4b81-a666-03e5abc1685b service neutron] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:04.148706 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1c97b8-bfdf-4b81-a666-03e5abc1685b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:04.160322 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1c97b8-bfdf-4b81-a666-03e5abc1685b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:04.164416 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1c97b8-bfdf-4b81-a666-03e5abc1685b service neutron] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:04.172041 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1097/2183] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:40:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:40:04.767202 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194e40de-0ae7-4400-b0b5-7c38ef99c42e None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:04.771236 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-194e40de-0ae7-4400-b0b5-7c38ef99c42e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:04.782749 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194e40de-0ae7-4400-b0b5-7c38ef99c42e None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:04.787742 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91812366-9698-4c76-b9ff-2b29e896ec82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:04.788384 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91812366-9698-4c76-b9ff-2b29e896ec82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:04.788667 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91812366-9698-4c76-b9ff-2b29e896ec82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:04.788945 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91812366-9698-4c76-b9ff-2b29e896ec82 admin admin] PATH_INFO: `/v3/projects/69a24f9ec9544eb788b0d756bb47ffb5` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:04.790110 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91812366-9698-4c76-b9ff-2b29e896ec82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=69a24f9ec9544eb788b0d756bb47ffb5)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:04.792018 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91812366-9698-4c76-b9ff-2b29e896ec82 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:04.929048 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-91812366-9698-4c76-b9ff-2b29e896ec82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69a24f9ec9544eb788b0d756bb47ffb5', 'request_id': 'req-91812366-9698-4c76-b9ff-2b29e896ec82'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:04.945203 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-91812366-9698-4c76-b9ff-2b29e896ec82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69a24f9ec9544eb788b0d756bb47ffb5', 'request_id': 'req-91812366-9698-4c76-b9ff-2b29e896ec82'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:05.039700 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-91812366-9698-4c76-b9ff-2b29e896ec82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69a24f9ec9544eb788b0d756bb47ffb5', 'request_id': 'req-91812366-9698-4c76-b9ff-2b29e896ec82'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:05.062971 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-91812366-9698-4c76-b9ff-2b29e896ec82 admin admin] The token cache is being invalidate because project 69a24f9ec9544eb788b0d756bb47ffb5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:40:05.062971 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-91812366-9698-4c76-b9ff-2b29e896ec82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:05.062971 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-91812366-9698-4c76-b9ff-2b29e896ec82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:05.065099 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1087/2184] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:40:04 2022] DELETE /identity/v3/projects/69a24f9ec9544eb788b0d756bb47ffb5 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:40:08.422730 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f98cc6-ae82-42a5-9fd6-0e4d7bc7672c None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:08.424420 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f98cc6-ae82-42a5-9fd6-0e4d7bc7672c None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:08.424420 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f98cc6-ae82-42a5-9fd6-0e4d7bc7672c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:08.455048 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-e4f98cc6-ae82-42a5-9fd6-0e4d7bc7672c None None] MFA Rules not processed for user `21de3a85b2df4af2833a24ab02f477c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:40:08.456463 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-e4f98cc6-ae82-42a5-9fd6-0e4d7bc7672c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:08.509245 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:40:08.533416 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1098/2185] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:40:08 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:40:08.544611 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefd3ae1-c328-4b5a-a576-2b79dffdbfb5 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:08.545190 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefd3ae1-c328-4b5a-a576-2b79dffdbfb5 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:08.545190 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefd3ae1-c328-4b5a-a576-2b79dffdbfb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:08.566139 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-cefd3ae1-c328-4b5a-a576-2b79dffdbfb5 None None] MFA Rules not processed for user `21de3a85b2df4af2833a24ab02f477c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:40:08.567929 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-cefd3ae1-c328-4b5a-a576-2b79dffdbfb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:08.575021 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1088/2186] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:40:08 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:40:08.588458 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f98cc6-ae82-42a5-9fd6-0e4d7bc7672c None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:08.590688 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-e4f98cc6-ae82-42a5-9fd6-0e4d7bc7672c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:08.669906 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f98cc6-ae82-42a5-9fd6-0e4d7bc7672c None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:08.675179 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a12de0-d4e0-4f0d-bdb4-159e8eb7384c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a080c7ab9b684abca6828067f2572c3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:08.675895 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a12de0-d4e0-4f0d-bdb4-159e8eb7384c service nova] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:08.676212 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a12de0-d4e0-4f0d-bdb4-159e8eb7384c service nova] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:08.676605 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a12de0-d4e0-4f0d-bdb4-159e8eb7384c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:08.684724 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a12de0-d4e0-4f0d-bdb4-159e8eb7384c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:08.686364 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a12de0-d4e0-4f0d-bdb4-159e8eb7384c service nova] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:08.698198 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1099/2187] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:40:08 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:40:09.434412 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47982325-24c7-42ff-9374-3a6ddf9a8c23 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:09.434696 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47982325-24c7-42ff-9374-3a6ddf9a8c23 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:09.435018 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47982325-24c7-42ff-9374-3a6ddf9a8c23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:09.462691 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-47982325-24c7-42ff-9374-3a6ddf9a8c23 None None] MFA Rules not processed for user `21de3a85b2df4af2833a24ab02f477c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:40:09.463933 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-47982325-24c7-42ff-9374-3a6ddf9a8c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:09.484811 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1089/2188] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:40:09 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:40:17.604631 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a12de0-d4e0-4f0d-bdb4-159e8eb7384c service nova] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:17.607370 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-88a12de0-d4e0-4f0d-bdb4-159e8eb7384c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:17.693897 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a12de0-d4e0-4f0d-bdb4-159e8eb7384c service nova] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:17.699082 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f3f590-9f95-4bd1-919a-e83a313afdab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:17.700781 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f3f590-9f95-4bd1-919a-e83a313afdab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:17.701142 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f3f590-9f95-4bd1-919a-e83a313afdab admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:17.701917 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f3f590-9f95-4bd1-919a-e83a313afdab admin admin] PATH_INFO: `/v3/users/048a8dccdce64e09bc87c65683ee34ef` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:17.703158 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f3f590-9f95-4bd1-919a-e83a313afdab admin admin] RBAC: Authorizing `identity:delete_user(user_id=048a8dccdce64e09bc87c65683ee34ef)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:17.705227 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f3f590-9f95-4bd1-919a-e83a313afdab admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:17.919675 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-f9f3f590-9f95-4bd1-919a-e83a313afdab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '048a8dccdce64e09bc87c65683ee34ef'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:18.006245 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-f9f3f590-9f95-4bd1-919a-e83a313afdab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '048a8dccdce64e09bc87c65683ee34ef'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:18.006893 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-f9f3f590-9f95-4bd1-919a-e83a313afdab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '048a8dccdce64e09bc87c65683ee34ef'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:18.007511 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-f9f3f590-9f95-4bd1-919a-e83a313afdab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '048a8dccdce64e09bc87c65683ee34ef'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:18.016852 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-f9f3f590-9f95-4bd1-919a-e83a313afdab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '048a8dccdce64e09bc87c65683ee34ef'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:18.024114 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:40:18.035932 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1100/2189] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:40:17 2022] DELETE /identity/v3/users/048a8dccdce64e09bc87c65683ee34ef => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 07:40:18.044108 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c18dfc-11e8-4fca-8260-fc250456a22e None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:18.044108 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c18dfc-11e8-4fca-8260-fc250456a22e None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:18.044108 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c18dfc-11e8-4fca-8260-fc250456a22e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:18.062432 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-27c18dfc-11e8-4fca-8260-fc250456a22e None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:40:18.063646 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-27c18dfc-11e8-4fca-8260-fc250456a22e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:18.124495 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1090/2190] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:40:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:40:18.134311 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f3f590-9f95-4bd1-919a-e83a313afdab admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:18.135786 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-f9f3f590-9f95-4bd1-919a-e83a313afdab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:18.194307 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f3f590-9f95-4bd1-919a-e83a313afdab admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:18.198089 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb84393-03b1-4391-b467-006be1170ef2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:18.198698 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb84393-03b1-4391-b467-006be1170ef2 service neutron] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:18.198974 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb84393-03b1-4391-b467-006be1170ef2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:18.199247 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb84393-03b1-4391-b467-006be1170ef2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:18.210258 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb84393-03b1-4391-b467-006be1170ef2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:18.211699 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb84393-03b1-4391-b467-006be1170ef2 service neutron] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:18.221324 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1101/2191] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:40:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:40:18.688576 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c18dfc-11e8-4fca-8260-fc250456a22e None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:18.690128 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-27c18dfc-11e8-4fca-8260-fc250456a22e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:18.703108 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c18dfc-11e8-4fca-8260-fc250456a22e None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:18.707818 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7719a244-7e97-4f5c-b4d6-3138695fde06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:18.708469 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7719a244-7e97-4f5c-b4d6-3138695fde06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:18.708752 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7719a244-7e97-4f5c-b4d6-3138695fde06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:18.709031 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7719a244-7e97-4f5c-b4d6-3138695fde06 admin admin] PATH_INFO: `/v3/projects/1c989f6976354d3aab8fe4ee30501c12` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:18.710211 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7719a244-7e97-4f5c-b4d6-3138695fde06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c989f6976354d3aab8fe4ee30501c12)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:18.712362 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7719a244-7e97-4f5c-b4d6-3138695fde06 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:18.725221 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:40:18.825136 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-7719a244-7e97-4f5c-b4d6-3138695fde06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c989f6976354d3aab8fe4ee30501c12', 'request_id': 'req-7719a244-7e97-4f5c-b4d6-3138695fde06'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:18.831455 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-7719a244-7e97-4f5c-b4d6-3138695fde06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c989f6976354d3aab8fe4ee30501c12', 'request_id': 'req-7719a244-7e97-4f5c-b4d6-3138695fde06'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:18.903136 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-7719a244-7e97-4f5c-b4d6-3138695fde06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c989f6976354d3aab8fe4ee30501c12', 'request_id': 'req-7719a244-7e97-4f5c-b4d6-3138695fde06'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:18.914301 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-7719a244-7e97-4f5c-b4d6-3138695fde06 admin admin] The token cache is being invalidate because project 1c989f6976354d3aab8fe4ee30501c12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:40:18.914590 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-7719a244-7e97-4f5c-b4d6-3138695fde06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:18.915211 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-7719a244-7e97-4f5c-b4d6-3138695fde06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:18.916684 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1091/2192] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:40:18 2022] DELETE /identity/v3/projects/1c989f6976354d3aab8fe4ee30501c12 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:40:18.921195 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb84393-03b1-4391-b467-006be1170ef2 service neutron] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:18.931128 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb84393-03b1-4391-b467-006be1170ef2 service neutron] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:18.988518 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f037af7f-35c9-4156-92f8-291d41aa354f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:18.992296 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f037af7f-35c9-4156-92f8-291d41aa354f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:18.992296 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f037af7f-35c9-4156-92f8-291d41aa354f admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:18.992296 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f037af7f-35c9-4156-92f8-291d41aa354f admin admin] PATH_INFO: `/v3/users/25474b1626db4592b948fb17977d3479` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:18.992296 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f037af7f-35c9-4156-92f8-291d41aa354f admin admin] RBAC: Authorizing `identity:delete_user(user_id=25474b1626db4592b948fb17977d3479)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:18.992753 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f037af7f-35c9-4156-92f8-291d41aa354f admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:19.150308 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-f037af7f-35c9-4156-92f8-291d41aa354f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25474b1626db4592b948fb17977d3479'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:19.217173 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-f037af7f-35c9-4156-92f8-291d41aa354f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25474b1626db4592b948fb17977d3479'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:19.217766 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-f037af7f-35c9-4156-92f8-291d41aa354f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25474b1626db4592b948fb17977d3479'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:19.218405 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-f037af7f-35c9-4156-92f8-291d41aa354f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25474b1626db4592b948fb17977d3479'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:19.225867 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-f037af7f-35c9-4156-92f8-291d41aa354f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25474b1626db4592b948fb17977d3479'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:19.244279 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1102/2193] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:40:18 2022] DELETE /identity/v3/users/25474b1626db4592b948fb17977d3479 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:40:19.641791 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7719a244-7e97-4f5c-b4d6-3138695fde06 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:19.643349 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-7719a244-7e97-4f5c-b4d6-3138695fde06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:19.659560 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7719a244-7e97-4f5c-b4d6-3138695fde06 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:19.723481 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0b7330-3d9f-4cae-8cfc-c6f041d1b08c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:19.724196 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0b7330-3d9f-4cae-8cfc-c6f041d1b08c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:19.724478 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0b7330-3d9f-4cae-8cfc-c6f041d1b08c admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:19.724751 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0b7330-3d9f-4cae-8cfc-c6f041d1b08c admin admin] PATH_INFO: `/v3/projects/5005fe9692f5404c8f9482df9dc6a920` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:19.725904 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0b7330-3d9f-4cae-8cfc-c6f041d1b08c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5005fe9692f5404c8f9482df9dc6a920)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:19.727880 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0b7330-3d9f-4cae-8cfc-c6f041d1b08c admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:19.824807 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-7d0b7330-3d9f-4cae-8cfc-c6f041d1b08c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5005fe9692f5404c8f9482df9dc6a920', 'request_id': 'req-7d0b7330-3d9f-4cae-8cfc-c6f041d1b08c'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:19.831602 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-7d0b7330-3d9f-4cae-8cfc-c6f041d1b08c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5005fe9692f5404c8f9482df9dc6a920', 'request_id': 'req-7d0b7330-3d9f-4cae-8cfc-c6f041d1b08c'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:19.900280 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-7d0b7330-3d9f-4cae-8cfc-c6f041d1b08c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5005fe9692f5404c8f9482df9dc6a920', 'request_id': 'req-7d0b7330-3d9f-4cae-8cfc-c6f041d1b08c'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:19.911164 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-7d0b7330-3d9f-4cae-8cfc-c6f041d1b08c admin admin] The token cache is being invalidate because project 5005fe9692f5404c8f9482df9dc6a920 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:40:19.911455 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-7d0b7330-3d9f-4cae-8cfc-c6f041d1b08c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:19.912076 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-7d0b7330-3d9f-4cae-8cfc-c6f041d1b08c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:19.913554 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1092/2194] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:40:19 2022] DELETE /identity/v3/projects/5005fe9692f5404c8f9482df9dc6a920 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:40:19.918631 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29490e8-71ae-4bfd-9c5d-3467e5685eb5 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:19.921234 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29490e8-71ae-4bfd-9c5d-3467e5685eb5 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:19.921234 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29490e8-71ae-4bfd-9c5d-3467e5685eb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:19.940717 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-d29490e8-71ae-4bfd-9c5d-3467e5685eb5 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:40:19.941955 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-d29490e8-71ae-4bfd-9c5d-3467e5685eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:19.950693 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:40:20.002788 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1103/2195] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:40:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:40:20.010745 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c380952a-461b-4cfb-b66f-4186694c9eca None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:20.011036 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c380952a-461b-4cfb-b66f-4186694c9eca None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:20.011344 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c380952a-461b-4cfb-b66f-4186694c9eca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:20.032098 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-c380952a-461b-4cfb-b66f-4186694c9eca None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:40:20.033342 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-c380952a-461b-4cfb-b66f-4186694c9eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:20.041491 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1093/2196] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:40:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:40:20.047097 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29490e8-71ae-4bfd-9c5d-3467e5685eb5 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:20.057474 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29490e8-71ae-4bfd-9c5d-3467e5685eb5 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:20.062867 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2d53cf-b0de-4338-a06a-4ee599d34860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:20.063505 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2d53cf-b0de-4338-a06a-4ee599d34860 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:20.063791 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2d53cf-b0de-4338-a06a-4ee599d34860 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:20.064068 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2d53cf-b0de-4338-a06a-4ee599d34860 admin admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:20.064714 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2d53cf-b0de-4338-a06a-4ee599d34860 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:40:20.065026 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2d53cf-b0de-4338-a06a-4ee599d34860 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:20.066666 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2d53cf-b0de-4338-a06a-4ee599d34860 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:20.081861 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1104/2197] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 15 07:40:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:40:20.082921 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c380952a-461b-4cfb-b66f-4186694c9eca None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:20.092345 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c380952a-461b-4cfb-b66f-4186694c9eca None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:20.097144 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98133f0-f180-4f18-8a71-202014e2d57d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:20.097781 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98133f0-f180-4f18-8a71-202014e2d57d admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:20.098067 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98133f0-f180-4f18-8a71-202014e2d57d admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:20.098333 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98133f0-f180-4f18-8a71-202014e2d57d admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:20.098979 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98133f0-f180-4f18-8a71-202014e2d57d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:20.100944 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98133f0-f180-4f18-8a71-202014e2d57d admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:20.187520 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1094/2198] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:40:20 2022] POST /identity/v3/projects => generated 387 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 07:40:20.191621 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2d53cf-b0de-4338-a06a-4ee599d34860 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:20.201723 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2d53cf-b0de-4338-a06a-4ee599d34860 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:20.261201 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27ad438-3bdd-4fa2-927a-b8a159997b8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:20.261849 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27ad438-3bdd-4fa2-927a-b8a159997b8d admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:20.262143 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27ad438-3bdd-4fa2-927a-b8a159997b8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:20.262410 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27ad438-3bdd-4fa2-927a-b8a159997b8d admin admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:20.263383 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27ad438-3bdd-4fa2-927a-b8a159997b8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:20.265504 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27ad438-3bdd-4fa2-927a-b8a159997b8d admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:20.420467 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1105/2199] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:40:20 2022] POST /identity/v3/users => generated 294 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:40:20.421939 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98133f0-f180-4f18-8a71-202014e2d57d admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:20.427841 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:40:20.432898 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98133f0-f180-4f18-8a71-202014e2d57d admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:20.439436 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d5d8de-5627-4199-862f-d9c8ed94f8c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:20.439763 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d5d8de-5627-4199-862f-d9c8ed94f8c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:20.440120 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d5d8de-5627-4199-862f-d9c8ed94f8c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:20.440468 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d5d8de-5627-4199-862f-d9c8ed94f8c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:20.441321 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d5d8de-5627-4199-862f-d9c8ed94f8c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:20.444032 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d5d8de-5627-4199-862f-d9c8ed94f8c8 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:20.458538 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1095/2200] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:40:20 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:40:20.459698 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27ad438-3bdd-4fa2-927a-b8a159997b8d admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:20.471968 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27ad438-3bdd-4fa2-927a-b8a159997b8d admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:20.480158 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4defd6f5-c8d8-4377-b5a8-ec684399f96c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:20.480912 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4defd6f5-c8d8-4377-b5a8-ec684399f96c admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:20.481190 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4defd6f5-c8d8-4377-b5a8-ec684399f96c admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:20.481465 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4defd6f5-c8d8-4377-b5a8-ec684399f96c admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:20.483740 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4defd6f5-c8d8-4377-b5a8-ec684399f96c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:20.485064 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4defd6f5-c8d8-4377-b5a8-ec684399f96c admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:20.502114 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1106/2201] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:40:20 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:40:20.502437 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d5d8de-5627-4199-862f-d9c8ed94f8c8 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:20.512897 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d5d8de-5627-4199-862f-d9c8ed94f8c8 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:20.519800 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e46d1c4-c89c-4941-97c1-56fa33f4aa92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:20.520498 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e46d1c4-c89c-4941-97c1-56fa33f4aa92 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:20.520784 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e46d1c4-c89c-4941-97c1-56fa33f4aa92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:20.521060 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e46d1c4-c89c-4941-97c1-56fa33f4aa92 admin admin] PATH_INFO: `/v3/projects/faa49ac1bf5d40c4b79cc5e6c6474cf9/users/81d3964426ce4621a7230905b7fc625c/roles/053e12af84e943e699460a9d76c2072b` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:20.540014 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e46d1c4-c89c-4941-97c1-56fa33f4aa92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=faa49ac1bf5d40c4b79cc5e6c6474cf9, user_id=81d3964426ce4621a7230905b7fc625c, role_id=053e12af84e943e699460a9d76c2072b)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:20.543033 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e46d1c4-c89c-4941-97c1-56fa33f4aa92 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:20.654830 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1096/2202] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:40:20 2022] PUT /identity/v3/projects/faa49ac1bf5d40c4b79cc5e6c6474cf9/users/81d3964426ce4621a7230905b7fc625c/roles/053e12af84e943e699460a9d76c2072b => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:40:20.670349 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d26a72-7c34-40e3-8f0d-5f31d7ff9c1b None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:20.670349 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d26a72-7c34-40e3-8f0d-5f31d7ff9c1b None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:20.670939 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d26a72-7c34-40e3-8f0d-5f31d7ff9c1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:20.703750 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-85d26a72-7c34-40e3-8f0d-5f31d7ff9c1b None None] MFA Rules not processed for user `81d3964426ce4621a7230905b7fc625c`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:40:20.705037 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-85d26a72-7c34-40e3-8f0d-5f31d7ff9c1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:20.787927 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1107/2203] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:40:20 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:40:20.792227 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e46d1c4-c89c-4941-97c1-56fa33f4aa92 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:20.856703 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e46d1c4-c89c-4941-97c1-56fa33f4aa92 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:20.861596 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eb0bee-ae3c-4985-986a-3a8815b20e0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:20.862258 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eb0bee-ae3c-4985-986a-3a8815b20e0f admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:20.862629 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eb0bee-ae3c-4985-986a-3a8815b20e0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:20.862891 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eb0bee-ae3c-4985-986a-3a8815b20e0f admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:20.863544 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67eb0bee-ae3c-4985-986a-3a8815b20e0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:20.865493 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67eb0bee-ae3c-4985-986a-3a8815b20e0f admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:20.985179 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1097/2204] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:40:20 2022] POST /identity/v3/projects => generated 387 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:40:20.989522 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d26a72-7c34-40e3-8f0d-5f31d7ff9c1b None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:21.001494 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d26a72-7c34-40e3-8f0d-5f31d7ff9c1b None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:21.006188 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:40:21.059492 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3465222-e1b8-49ca-8445-a53556d55c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:21.059911 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3465222-e1b8-49ca-8445-a53556d55c8a admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:21.060711 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3465222-e1b8-49ca-8445-a53556d55c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:21.061042 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3465222-e1b8-49ca-8445-a53556d55c8a admin admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:21.062263 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3465222-e1b8-49ca-8445-a53556d55c8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:21.067119 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3465222-e1b8-49ca-8445-a53556d55c8a admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:21.163380 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1108/2205] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:40:20 2022] POST /identity/v3/users => generated 300 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:40:21.164555 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eb0bee-ae3c-4985-986a-3a8815b20e0f admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:21.188626 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eb0bee-ae3c-4985-986a-3a8815b20e0f admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:21.190473 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85803ca-e4a0-40b4-92e6-1f284c5305f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:21.192029 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85803ca-e4a0-40b4-92e6-1f284c5305f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:21.192412 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85803ca-e4a0-40b4-92e6-1f284c5305f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:21.192785 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85803ca-e4a0-40b4-92e6-1f284c5305f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:21.193927 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85803ca-e4a0-40b4-92e6-1f284c5305f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:21.196290 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85803ca-e4a0-40b4-92e6-1f284c5305f0 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:21.219704 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1098/2206] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:40:21 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:40:21.219857 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3465222-e1b8-49ca-8445-a53556d55c8a admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:21.229692 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3465222-e1b8-49ca-8445-a53556d55c8a admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:21.238846 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f5b05a-2f88-4f86-bb8a-f762ece0ad3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:21.239629 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f5b05a-2f88-4f86-bb8a-f762ece0ad3c admin admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:21.240286 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f5b05a-2f88-4f86-bb8a-f762ece0ad3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:21.245805 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f5b05a-2f88-4f86-bb8a-f762ece0ad3c admin admin] PATH_INFO: `/v3/projects/124373b58ff34e87b78b6860e27a222b/users/2c0e2661aa404781b4bb163c2ef3a972/roles/6bbb643e0d8d4bc0a9b27c26edf59216` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:21.260138 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f5b05a-2f88-4f86-bb8a-f762ece0ad3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=124373b58ff34e87b78b6860e27a222b, user_id=2c0e2661aa404781b4bb163c2ef3a972, role_id=6bbb643e0d8d4bc0a9b27c26edf59216)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:21.264160 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f5b05a-2f88-4f86-bb8a-f762ece0ad3c admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:21.338525 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1109/2207] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:40:21 2022] PUT /identity/v3/projects/124373b58ff34e87b78b6860e27a222b/users/2c0e2661aa404781b4bb163c2ef3a972/roles/6bbb643e0d8d4bc0a9b27c26edf59216 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:40:21.343384 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85803ca-e4a0-40b4-92e6-1f284c5305f0 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:21.354303 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85803ca-e4a0-40b4-92e6-1f284c5305f0 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:21.412188 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9c4a88-8167-49ca-8916-7f4a280f835c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:21.412809 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9c4a88-8167-49ca-8916-7f4a280f835c admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:21.414972 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9c4a88-8167-49ca-8916-7f4a280f835c admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:21.414972 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9c4a88-8167-49ca-8916-7f4a280f835c admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:21.414972 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9c4a88-8167-49ca-8916-7f4a280f835c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:21.415646 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9c4a88-8167-49ca-8916-7f4a280f835c admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:21.425995 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1099/2208] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:40:21 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:40:21.430182 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f5b05a-2f88-4f86-bb8a-f762ece0ad3c admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:21.440838 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f5b05a-2f88-4f86-bb8a-f762ece0ad3c admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:21.445596 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7a057a-d2ac-4d95-add5-beb2cb900973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:21.446265 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7a057a-d2ac-4d95-add5-beb2cb900973 admin admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:21.446547 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7a057a-d2ac-4d95-add5-beb2cb900973 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:21.446826 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7a057a-d2ac-4d95-add5-beb2cb900973 admin admin] PATH_INFO: `/v3/projects/124373b58ff34e87b78b6860e27a222b/users/2c0e2661aa404781b4bb163c2ef3a972/roles/6bbb643e0d8d4bc0a9b27c26edf59216` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:21.448801 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7a057a-d2ac-4d95-add5-beb2cb900973 admin admin] RBAC: Authorizing `identity:create_grant(project_id=124373b58ff34e87b78b6860e27a222b, user_id=2c0e2661aa404781b4bb163c2ef3a972, role_id=6bbb643e0d8d4bc0a9b27c26edf59216)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:21.452267 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7a057a-d2ac-4d95-add5-beb2cb900973 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:21.462637 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1110/2209] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:40:21 2022] PUT /identity/v3/projects/124373b58ff34e87b78b6860e27a222b/users/2c0e2661aa404781b4bb163c2ef3a972/roles/6bbb643e0d8d4bc0a9b27c26edf59216 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:40:21.473721 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dd4148-7184-4d43-a0f6-6d574e7ec944 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:21.474011 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dd4148-7184-4d43-a0f6-6d574e7ec944 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:21.474288 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dd4148-7184-4d43-a0f6-6d574e7ec944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:21.502506 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-c1dd4148-7184-4d43-a0f6-6d574e7ec944 None None] MFA Rules not processed for user `2c0e2661aa404781b4bb163c2ef3a972`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:40:21.503725 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-c1dd4148-7184-4d43-a0f6-6d574e7ec944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:21.534716 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:40:21.579034 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1100/2210] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:40:21 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:40:21.843208 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7a057a-d2ac-4d95-add5-beb2cb900973 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:21.845930 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-3e7a057a-d2ac-4d95-add5-beb2cb900973 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:21.967405 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7a057a-d2ac-4d95-add5-beb2cb900973 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:21.972146 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786f0d84-18d6-4b8d-b67e-b3fe1b4cb72a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecf6642b7b84d3c9756bb41ab3349d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:21.973212 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786f0d84-18d6-4b8d-b67e-b3fe1b4cb72a service cinder] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:21.973663 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786f0d84-18d6-4b8d-b67e-b3fe1b4cb72a service cinder] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:21.974252 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786f0d84-18d6-4b8d-b67e-b3fe1b4cb72a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:21.988934 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786f0d84-18d6-4b8d-b67e-b3fe1b4cb72a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:21.990487 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786f0d84-18d6-4b8d-b67e-b3fe1b4cb72a service cinder] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:22.006475 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1111/2211] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:40:21 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:40:23.489276 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dd4148-7184-4d43-a0f6-6d574e7ec944 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:23.552841 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dd4148-7184-4d43-a0f6-6d574e7ec944 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:23.557418 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55776b5-c041-44b2-935b-45b369fa6fea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:23.558082 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55776b5-c041-44b2-935b-45b369fa6fea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:23.558371 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55776b5-c041-44b2-935b-45b369fa6fea admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:23.558650 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55776b5-c041-44b2-935b-45b369fa6fea admin admin] PATH_INFO: `/v3/users/81d3964426ce4621a7230905b7fc625c` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:23.559773 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55776b5-c041-44b2-935b-45b369fa6fea admin admin] RBAC: Authorizing `identity:delete_user(user_id=81d3964426ce4621a7230905b7fc625c)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:23.562303 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55776b5-c041-44b2-935b-45b369fa6fea admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:23.717796 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-e55776b5-c041-44b2-935b-45b369fa6fea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81d3964426ce4621a7230905b7fc625c'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:23.718479 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-e55776b5-c041-44b2-935b-45b369fa6fea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81d3964426ce4621a7230905b7fc625c'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:23.800949 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-e55776b5-c041-44b2-935b-45b369fa6fea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81d3964426ce4621a7230905b7fc625c'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:23.801601 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-e55776b5-c041-44b2-935b-45b369fa6fea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81d3964426ce4621a7230905b7fc625c'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:23.809210 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-e55776b5-c041-44b2-935b-45b369fa6fea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81d3964426ce4621a7230905b7fc625c'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:23.832140 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1101/2212] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:40:23 2022] DELETE /identity/v3/users/81d3964426ce4621a7230905b7fc625c => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:40:23.836587 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55320399-a1d1-4fb5-b065-50cd524a9c63 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:23.837031 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55320399-a1d1-4fb5-b065-50cd524a9c63 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:23.837153 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55320399-a1d1-4fb5-b065-50cd524a9c63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:23.855994 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-55320399-a1d1-4fb5-b065-50cd524a9c63 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:40:23.857186 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-55320399-a1d1-4fb5-b065-50cd524a9c63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:23.917471 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1112/2213] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:40:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:40:23.927866 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55776b5-c041-44b2-935b-45b369fa6fea admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:23.929280 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-e55776b5-c041-44b2-935b-45b369fa6fea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:23.978667 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:40:23.993244 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55776b5-c041-44b2-935b-45b369fa6fea admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:23.996501 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2aac98-4226-4a72-80db-3048d31dc93e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:23.997220 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2aac98-4226-4a72-80db-3048d31dc93e service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:23.997596 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2aac98-4226-4a72-80db-3048d31dc93e service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:23.998027 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2aac98-4226-4a72-80db-3048d31dc93e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:24.007912 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2aac98-4226-4a72-80db-3048d31dc93e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:24.009446 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2aac98-4226-4a72-80db-3048d31dc93e service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:24.020534 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1102/2214] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:40:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 07:40:24.436957 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55320399-a1d1-4fb5-b065-50cd524a9c63 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:24.438545 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-55320399-a1d1-4fb5-b065-50cd524a9c63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:24.451355 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:40:24.455631 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55320399-a1d1-4fb5-b065-50cd524a9c63 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:24.460372 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb559e4-06ba-4031-8339-92dafc7c1095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:24.461052 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb559e4-06ba-4031-8339-92dafc7c1095 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:24.461330 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb559e4-06ba-4031-8339-92dafc7c1095 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:24.461634 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb559e4-06ba-4031-8339-92dafc7c1095 admin admin] PATH_INFO: `/v3/projects/faa49ac1bf5d40c4b79cc5e6c6474cf9` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:24.462767 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb559e4-06ba-4031-8339-92dafc7c1095 admin admin] RBAC: Authorizing `identity:delete_project(project_id=faa49ac1bf5d40c4b79cc5e6c6474cf9)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:24.464683 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb559e4-06ba-4031-8339-92dafc7c1095 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:24.638180 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-eeb559e4-06ba-4031-8339-92dafc7c1095 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'faa49ac1bf5d40c4b79cc5e6c6474cf9', 'request_id': 'req-eeb559e4-06ba-4031-8339-92dafc7c1095'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:24.644841 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-eeb559e4-06ba-4031-8339-92dafc7c1095 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'faa49ac1bf5d40c4b79cc5e6c6474cf9', 'request_id': 'req-eeb559e4-06ba-4031-8339-92dafc7c1095'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:24.769726 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-eeb559e4-06ba-4031-8339-92dafc7c1095 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'faa49ac1bf5d40c4b79cc5e6c6474cf9', 'request_id': 'req-eeb559e4-06ba-4031-8339-92dafc7c1095'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:24.782015 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-eeb559e4-06ba-4031-8339-92dafc7c1095 admin admin] The token cache is being invalidate because project faa49ac1bf5d40c4b79cc5e6c6474cf9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:40:24.782312 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-eeb559e4-06ba-4031-8339-92dafc7c1095 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:24.782953 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-eeb559e4-06ba-4031-8339-92dafc7c1095 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:24.784491 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1113/2215] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:40:24 2022] DELETE /identity/v3/projects/faa49ac1bf5d40c4b79cc5e6c6474cf9 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 07:40:24.788714 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2aac98-4226-4a72-80db-3048d31dc93e service neutron] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:24.800914 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2aac98-4226-4a72-80db-3048d31dc93e service neutron] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:24.869260 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c78763-301b-42c1-9c7d-3797d516bbbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:24.869945 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c78763-301b-42c1-9c7d-3797d516bbbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:24.870229 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c78763-301b-42c1-9c7d-3797d516bbbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:24.870502 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c78763-301b-42c1-9c7d-3797d516bbbe admin admin] PATH_INFO: `/v3/users/2c0e2661aa404781b4bb163c2ef3a972` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:24.871628 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c78763-301b-42c1-9c7d-3797d516bbbe admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c0e2661aa404781b4bb163c2ef3a972)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:24.873896 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c78763-301b-42c1-9c7d-3797d516bbbe admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:25.046906 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-e1c78763-301b-42c1-9c7d-3797d516bbbe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c0e2661aa404781b4bb163c2ef3a972'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:25.047617 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-e1c78763-301b-42c1-9c7d-3797d516bbbe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c0e2661aa404781b4bb163c2ef3a972'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:25.109688 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-e1c78763-301b-42c1-9c7d-3797d516bbbe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c0e2661aa404781b4bb163c2ef3a972'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:25.110360 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-e1c78763-301b-42c1-9c7d-3797d516bbbe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c0e2661aa404781b4bb163c2ef3a972'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:25.117936 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-e1c78763-301b-42c1-9c7d-3797d516bbbe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c0e2661aa404781b4bb163c2ef3a972'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:25.135129 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1103/2216] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:40:24 2022] DELETE /identity/v3/users/2c0e2661aa404781b4bb163c2ef3a972 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:40:25.591159 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb559e4-06ba-4031-8339-92dafc7c1095 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:25.593215 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-eeb559e4-06ba-4031-8339-92dafc7c1095 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:25.606311 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb559e4-06ba-4031-8339-92dafc7c1095 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:25.668965 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ee75cd-2635-454e-84cb-196c5aa2df99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:25.669888 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ee75cd-2635-454e-84cb-196c5aa2df99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:25.670531 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ee75cd-2635-454e-84cb-196c5aa2df99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:25.670708 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ee75cd-2635-454e-84cb-196c5aa2df99 admin admin] PATH_INFO: `/v3/projects/124373b58ff34e87b78b6860e27a222b` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:25.671911 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ee75cd-2635-454e-84cb-196c5aa2df99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=124373b58ff34e87b78b6860e27a222b)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:25.674088 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ee75cd-2635-454e-84cb-196c5aa2df99 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:25.779994 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-13ee75cd-2635-454e-84cb-196c5aa2df99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '124373b58ff34e87b78b6860e27a222b', 'request_id': 'req-13ee75cd-2635-454e-84cb-196c5aa2df99'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:25.786310 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-13ee75cd-2635-454e-84cb-196c5aa2df99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '124373b58ff34e87b78b6860e27a222b', 'request_id': 'req-13ee75cd-2635-454e-84cb-196c5aa2df99'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:25.866924 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-13ee75cd-2635-454e-84cb-196c5aa2df99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '124373b58ff34e87b78b6860e27a222b', 'request_id': 'req-13ee75cd-2635-454e-84cb-196c5aa2df99'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:25.878903 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-13ee75cd-2635-454e-84cb-196c5aa2df99 admin admin] The token cache is being invalidate because project 124373b58ff34e87b78b6860e27a222b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:40:25.879198 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-13ee75cd-2635-454e-84cb-196c5aa2df99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:25.879866 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-13ee75cd-2635-454e-84cb-196c5aa2df99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:25.881363 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1114/2217] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:40:25 2022] DELETE /identity/v3/projects/124373b58ff34e87b78b6860e27a222b => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:40:25.886102 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32553dfb-7f0b-4bdb-8067-97f2fdcdec53 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:25.886553 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32553dfb-7f0b-4bdb-8067-97f2fdcdec53 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:25.886783 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32553dfb-7f0b-4bdb-8067-97f2fdcdec53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:25.906656 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-32553dfb-7f0b-4bdb-8067-97f2fdcdec53 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:40:25.907848 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-32553dfb-7f0b-4bdb-8067-97f2fdcdec53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:25.980936 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1104/2218] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:40:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:40:25.989902 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbab620-594c-48d1-bac5-1bf49135254e None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:25.990194 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbab620-594c-48d1-bac5-1bf49135254e None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:25.990477 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbab620-594c-48d1-bac5-1bf49135254e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:26.011570 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-1bbab620-594c-48d1-bac5-1bf49135254e None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:40:26.012820 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-1bbab620-594c-48d1-bac5-1bf49135254e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:26.022443 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1115/2219] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:40:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:40:26.026768 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32553dfb-7f0b-4bdb-8067-97f2fdcdec53 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:26.033982 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:40:26.038484 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32553dfb-7f0b-4bdb-8067-97f2fdcdec53 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:26.043314 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a71b44-1e28-4965-aa37-45f75bb44ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:26.043984 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a71b44-1e28-4965-aa37-45f75bb44ad5 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:26.044287 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a71b44-1e28-4965-aa37-45f75bb44ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:26.044563 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a71b44-1e28-4965-aa37-45f75bb44ad5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:26.046320 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a71b44-1e28-4965-aa37-45f75bb44ad5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:40:26.046635 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a71b44-1e28-4965-aa37-45f75bb44ad5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:26.048241 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a71b44-1e28-4965-aa37-45f75bb44ad5 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:26.061108 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1105/2220] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 15 07:40:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:40:26.065698 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbab620-594c-48d1-bac5-1bf49135254e None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:26.075391 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbab620-594c-48d1-bac5-1bf49135254e None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:26.080461 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370f6a74-8bd8-451b-9934-ef174a99740c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:26.081692 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370f6a74-8bd8-451b-9934-ef174a99740c admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:26.081692 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370f6a74-8bd8-451b-9934-ef174a99740c admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:26.081886 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370f6a74-8bd8-451b-9934-ef174a99740c admin admin] PATH_INFO: `/v3/projects` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:26.082475 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370f6a74-8bd8-451b-9934-ef174a99740c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:26.084508 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370f6a74-8bd8-451b-9934-ef174a99740c admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:26.095729 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:40:26.202354 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1116/2221] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:40:26 2022] POST /identity/v3/projects => generated 375 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:40:26.206675 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a71b44-1e28-4965-aa37-45f75bb44ad5 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:26.217391 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a71b44-1e28-4965-aa37-45f75bb44ad5 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:26.273023 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fa6d4a-0820-4826-8626-73765fb1f491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:26.273661 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fa6d4a-0820-4826-8626-73765fb1f491 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:26.273975 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fa6d4a-0820-4826-8626-73765fb1f491 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:26.274252 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fa6d4a-0820-4826-8626-73765fb1f491 admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:26.274892 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fa6d4a-0820-4826-8626-73765fb1f491 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:26.276851 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fa6d4a-0820-4826-8626-73765fb1f491 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:26.415958 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1106/2222] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:40:26 2022] POST /identity/v3/users => generated 288 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:40:26.417044 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370f6a74-8bd8-451b-9934-ef174a99740c admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:26.428171 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370f6a74-8bd8-451b-9934-ef174a99740c admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:26.432654 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3182935d-0eec-4782-bb75-2da8fe0dbc23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:26.433257 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3182935d-0eec-4782-bb75-2da8fe0dbc23 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:26.433564 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3182935d-0eec-4782-bb75-2da8fe0dbc23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:26.435225 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3182935d-0eec-4782-bb75-2da8fe0dbc23 admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:26.436177 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3182935d-0eec-4782-bb75-2da8fe0dbc23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:26.437913 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3182935d-0eec-4782-bb75-2da8fe0dbc23 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:26.450466 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1117/2223] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:40:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:40:26.453250 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fa6d4a-0820-4826-8626-73765fb1f491 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:26.463517 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fa6d4a-0820-4826-8626-73765fb1f491 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:26.470546 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1443ecdc-90d6-4b0c-bc93-297c2e5003f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:26.471153 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1443ecdc-90d6-4b0c-bc93-297c2e5003f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:26.471431 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1443ecdc-90d6-4b0c-bc93-297c2e5003f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:26.471700 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1443ecdc-90d6-4b0c-bc93-297c2e5003f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:26.472340 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1443ecdc-90d6-4b0c-bc93-297c2e5003f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:26.473996 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1443ecdc-90d6-4b0c-bc93-297c2e5003f4 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:26.486179 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1107/2224] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:40:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:40:26.490440 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3182935d-0eec-4782-bb75-2da8fe0dbc23 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:26.500651 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3182935d-0eec-4782-bb75-2da8fe0dbc23 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:26.505552 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76df7ea-8f56-4e54-886b-e166b9f89e44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:26.509354 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76df7ea-8f56-4e54-886b-e166b9f89e44 admin admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:26.509354 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76df7ea-8f56-4e54-886b-e166b9f89e44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:26.509354 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76df7ea-8f56-4e54-886b-e166b9f89e44 admin admin] PATH_INFO: `/v3/projects/e0730e401f844b55a78df53dba8e4b5c/users/82e55686994a4e6b86096cc21c52085d/roles/053e12af84e943e699460a9d76c2072b` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:26.523111 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76df7ea-8f56-4e54-886b-e166b9f89e44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0730e401f844b55a78df53dba8e4b5c, user_id=82e55686994a4e6b86096cc21c52085d, role_id=053e12af84e943e699460a9d76c2072b)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:26.525805 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76df7ea-8f56-4e54-886b-e166b9f89e44 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:26.621704 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1118/2225] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:40:26 2022] PUT /identity/v3/projects/e0730e401f844b55a78df53dba8e4b5c/users/82e55686994a4e6b86096cc21c52085d/roles/053e12af84e943e699460a9d76c2072b => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:40:26.691719 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed1354e-66c6-416d-8396-a769f38640b0 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:26.691931 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed1354e-66c6-416d-8396-a769f38640b0 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:26.692214 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed1354e-66c6-416d-8396-a769f38640b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:26.721731 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-eed1354e-66c6-416d-8396-a769f38640b0 None None] MFA Rules not processed for user `82e55686994a4e6b86096cc21c52085d`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:40:26.723083 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-eed1354e-66c6-416d-8396-a769f38640b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:26.794093 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1108/2226] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:40:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:40:26.805052 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76df7ea-8f56-4e54-886b-e166b9f89e44 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:26.806816 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-b76df7ea-8f56-4e54-886b-e166b9f89e44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:26.867168 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76df7ea-8f56-4e54-886b-e166b9f89e44 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:26.870839 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6dce5a-5cb5-4ab2-bb07-bf6a43802b22 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecf6642b7b84d3c9756bb41ab3349d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:26.871436 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6dce5a-5cb5-4ab2-bb07-bf6a43802b22 service cinder] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:26.871712 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6dce5a-5cb5-4ab2-bb07-bf6a43802b22 service cinder] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:26.871978 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6dce5a-5cb5-4ab2-bb07-bf6a43802b22 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:26.884792 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6dce5a-5cb5-4ab2-bb07-bf6a43802b22 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:26.886274 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6dce5a-5cb5-4ab2-bb07-bf6a43802b22 service cinder] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:26.896133 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1119/2227] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:40:26 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:40:28.734134 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b6a4fb-3e11-4945-affa-1162e2fc2595 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:28.734679 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b6a4fb-3e11-4945-affa-1162e2fc2595 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:28.734679 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b6a4fb-3e11-4945-affa-1162e2fc2595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:28.760850 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-35b6a4fb-3e11-4945-affa-1162e2fc2595 None None] MFA Rules not processed for user `82e55686994a4e6b86096cc21c52085d`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:40:28.762101 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-35b6a4fb-3e11-4945-affa-1162e2fc2595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:28.770023 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1109/2228] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:40:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:40:30.090343 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6dce5a-5cb5-4ab2-bb07-bf6a43802b22 service cinder] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:30.092388 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-ef6dce5a-5cb5-4ab2-bb07-bf6a43802b22 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:30.162776 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6dce5a-5cb5-4ab2-bb07-bf6a43802b22 service cinder] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:30.167747 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aefaf95-069e-4b8e-a2eb-928ad24ffbfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:30.168388 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aefaf95-069e-4b8e-a2eb-928ad24ffbfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:30.168685 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aefaf95-069e-4b8e-a2eb-928ad24ffbfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:30.168960 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aefaf95-069e-4b8e-a2eb-928ad24ffbfd admin admin] PATH_INFO: `/v3/users/a3587236bc2e40a8a0701930579e4eed` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:30.170232 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aefaf95-069e-4b8e-a2eb-928ad24ffbfd admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3587236bc2e40a8a0701930579e4eed)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:30.172189 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aefaf95-069e-4b8e-a2eb-928ad24ffbfd admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:30.391531 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-0aefaf95-069e-4b8e-a2eb-928ad24ffbfd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3587236bc2e40a8a0701930579e4eed'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:30.453222 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-0aefaf95-069e-4b8e-a2eb-928ad24ffbfd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3587236bc2e40a8a0701930579e4eed'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:30.453796 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-0aefaf95-069e-4b8e-a2eb-928ad24ffbfd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3587236bc2e40a8a0701930579e4eed'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:30.454407 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-0aefaf95-069e-4b8e-a2eb-928ad24ffbfd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3587236bc2e40a8a0701930579e4eed'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:30.461611 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:40:30.465154 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-0aefaf95-069e-4b8e-a2eb-928ad24ffbfd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3587236bc2e40a8a0701930579e4eed'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:30.484158 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1120/2229] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:40:30 2022] DELETE /identity/v3/users/a3587236bc2e40a8a0701930579e4eed => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:40:30.488647 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cf6e41-9df4-48f7-8851-649ffcd4ca02 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:30.489132 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cf6e41-9df4-48f7-8851-649ffcd4ca02 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:30.489245 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cf6e41-9df4-48f7-8851-649ffcd4ca02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:30.513122 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-86cf6e41-9df4-48f7-8851-649ffcd4ca02 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:40:30.514507 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-86cf6e41-9df4-48f7-8851-649ffcd4ca02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:30.564914 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:40:30.577383 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1110/2230] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:40:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:40:30.587156 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aefaf95-069e-4b8e-a2eb-928ad24ffbfd admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:30.588610 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-0aefaf95-069e-4b8e-a2eb-928ad24ffbfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:30.664244 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aefaf95-069e-4b8e-a2eb-928ad24ffbfd admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:30.668213 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3115ab74-89df-41f9-bc05-1dca53d46d8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:30.669006 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3115ab74-89df-41f9-bc05-1dca53d46d8f service neutron] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:30.669358 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3115ab74-89df-41f9-bc05-1dca53d46d8f service neutron] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:30.669737 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3115ab74-89df-41f9-bc05-1dca53d46d8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:30.680403 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3115ab74-89df-41f9-bc05-1dca53d46d8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:30.682167 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3115ab74-89df-41f9-bc05-1dca53d46d8f service neutron] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:30.692514 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1121/2231] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:40:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:40:31.183185 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cf6e41-9df4-48f7-8851-649ffcd4ca02 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:31.184827 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-86cf6e41-9df4-48f7-8851-649ffcd4ca02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:31.198276 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cf6e41-9df4-48f7-8851-649ffcd4ca02 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:31.202939 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a14aa5-22d0-4cfc-9a2a-db1fce8f46b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:31.203593 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a14aa5-22d0-4cfc-9a2a-db1fce8f46b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:31.203835 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a14aa5-22d0-4cfc-9a2a-db1fce8f46b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:31.204088 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a14aa5-22d0-4cfc-9a2a-db1fce8f46b0 admin admin] PATH_INFO: `/v3/projects/240918f14a0843abab182af6ff9b1e87` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:31.205230 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a14aa5-22d0-4cfc-9a2a-db1fce8f46b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=240918f14a0843abab182af6ff9b1e87)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:31.207169 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a14aa5-22d0-4cfc-9a2a-db1fce8f46b0 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:31.339496 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-85a14aa5-22d0-4cfc-9a2a-db1fce8f46b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '240918f14a0843abab182af6ff9b1e87', 'request_id': 'req-85a14aa5-22d0-4cfc-9a2a-db1fce8f46b0'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:31.345917 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-85a14aa5-22d0-4cfc-9a2a-db1fce8f46b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '240918f14a0843abab182af6ff9b1e87', 'request_id': 'req-85a14aa5-22d0-4cfc-9a2a-db1fce8f46b0'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:31.433200 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-85a14aa5-22d0-4cfc-9a2a-db1fce8f46b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '240918f14a0843abab182af6ff9b1e87', 'request_id': 'req-85a14aa5-22d0-4cfc-9a2a-db1fce8f46b0'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:31.444772 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-85a14aa5-22d0-4cfc-9a2a-db1fce8f46b0 admin admin] The token cache is being invalidate because project 240918f14a0843abab182af6ff9b1e87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:40:31.445062 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-85a14aa5-22d0-4cfc-9a2a-db1fce8f46b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:31.445739 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-85a14aa5-22d0-4cfc-9a2a-db1fce8f46b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:31.447220 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1111/2232] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:40:31 2022] DELETE /identity/v3/projects/240918f14a0843abab182af6ff9b1e87 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 07:40:31.451268 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3115ab74-89df-41f9-bc05-1dca53d46d8f service neutron] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:31.461299 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3115ab74-89df-41f9-bc05-1dca53d46d8f service neutron] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:31.519268 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe905458-0fa3-433d-aab3-8b0034b0cbcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:31.519881 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe905458-0fa3-433d-aab3-8b0034b0cbcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:31.520187 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe905458-0fa3-433d-aab3-8b0034b0cbcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:31.520457 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe905458-0fa3-433d-aab3-8b0034b0cbcc admin admin] PATH_INFO: `/v3/users/9cd2a5b64a034677afa512adb3299c36` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:31.521567 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe905458-0fa3-433d-aab3-8b0034b0cbcc admin admin] RBAC: Authorizing `identity:delete_user(user_id=9cd2a5b64a034677afa512adb3299c36)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:31.523571 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe905458-0fa3-433d-aab3-8b0034b0cbcc admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:31.740903 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-fe905458-0fa3-433d-aab3-8b0034b0cbcc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9cd2a5b64a034677afa512adb3299c36'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:31.829560 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-fe905458-0fa3-433d-aab3-8b0034b0cbcc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9cd2a5b64a034677afa512adb3299c36'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:31.830238 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-fe905458-0fa3-433d-aab3-8b0034b0cbcc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9cd2a5b64a034677afa512adb3299c36'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:31.830846 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-fe905458-0fa3-433d-aab3-8b0034b0cbcc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9cd2a5b64a034677afa512adb3299c36'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:31.838716 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-fe905458-0fa3-433d-aab3-8b0034b0cbcc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9cd2a5b64a034677afa512adb3299c36'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:31.852493 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:40:31.857958 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1122/2233] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:40:31 2022] DELETE /identity/v3/users/9cd2a5b64a034677afa512adb3299c36 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:40:32.319712 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a14aa5-22d0-4cfc-9a2a-db1fce8f46b0 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:32.321171 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-85a14aa5-22d0-4cfc-9a2a-db1fce8f46b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:32.336137 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a14aa5-22d0-4cfc-9a2a-db1fce8f46b0 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:32.399395 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6816a71-2b6c-4a96-a851-a19d823c7685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:32.400423 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6816a71-2b6c-4a96-a851-a19d823c7685 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:32.400706 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6816a71-2b6c-4a96-a851-a19d823c7685 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:32.400981 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6816a71-2b6c-4a96-a851-a19d823c7685 admin admin] PATH_INFO: `/v3/projects/602ac6a82f5742b98e2c0123f204301c` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:32.402178 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6816a71-2b6c-4a96-a851-a19d823c7685 admin admin] RBAC: Authorizing `identity:delete_project(project_id=602ac6a82f5742b98e2c0123f204301c)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:32.404138 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6816a71-2b6c-4a96-a851-a19d823c7685 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:32.547525 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-f6816a71-2b6c-4a96-a851-a19d823c7685 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '602ac6a82f5742b98e2c0123f204301c', 'request_id': 'req-f6816a71-2b6c-4a96-a851-a19d823c7685'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:32.557599 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-f6816a71-2b6c-4a96-a851-a19d823c7685 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '602ac6a82f5742b98e2c0123f204301c', 'request_id': 'req-f6816a71-2b6c-4a96-a851-a19d823c7685'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:32.704296 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-f6816a71-2b6c-4a96-a851-a19d823c7685 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '602ac6a82f5742b98e2c0123f204301c', 'request_id': 'req-f6816a71-2b6c-4a96-a851-a19d823c7685'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:32.717986 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-f6816a71-2b6c-4a96-a851-a19d823c7685 admin admin] The token cache is being invalidate because project 602ac6a82f5742b98e2c0123f204301c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:40:32.718420 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-f6816a71-2b6c-4a96-a851-a19d823c7685 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:32.719092 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-f6816a71-2b6c-4a96-a851-a19d823c7685 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:32.720604 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1112/2234] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:40:32 2022] DELETE /identity/v3/projects/602ac6a82f5742b98e2c0123f204301c => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:40:32.725718 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f087fc6-d94d-47e9-aa86-8c42f84b7c41 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:32.726360 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f087fc6-d94d-47e9-aa86-8c42f84b7c41 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:32.726360 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f087fc6-d94d-47e9-aa86-8c42f84b7c41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:32.750252 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-8f087fc6-d94d-47e9-aa86-8c42f84b7c41 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:40:32.751515 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-8f087fc6-d94d-47e9-aa86-8c42f84b7c41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:32.821407 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1123/2235] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:40:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 07:40:32.830265 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f1ba04-df53-441d-a9ba-1c700672a2da None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:32.830562 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f1ba04-df53-441d-a9ba-1c700672a2da None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:32.830840 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f1ba04-df53-441d-a9ba-1c700672a2da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:32.852764 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:40:32.859307 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-12f1ba04-df53-441d-a9ba-1c700672a2da None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:40:32.863757 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-12f1ba04-df53-441d-a9ba-1c700672a2da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:32.868584 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1113/2236] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:40:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:40:32.872982 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f087fc6-d94d-47e9-aa86-8c42f84b7c41 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:32.883977 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f087fc6-d94d-47e9-aa86-8c42f84b7c41 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:32.890979 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abae92f3-8a04-4823-98e6-400a087ac8d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:32.892212 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abae92f3-8a04-4823-98e6-400a087ac8d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:32.892554 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abae92f3-8a04-4823-98e6-400a087ac8d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:32.892930 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abae92f3-8a04-4823-98e6-400a087ac8d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:32.894720 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abae92f3-8a04-4823-98e6-400a087ac8d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:40:32.894720 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abae92f3-8a04-4823-98e6-400a087ac8d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:32.895726 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abae92f3-8a04-4823-98e6-400a087ac8d5 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:32.914758 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1124/2237] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 15 07:40:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:40:32.916296 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f1ba04-df53-441d-a9ba-1c700672a2da None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:32.927405 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f1ba04-df53-441d-a9ba-1c700672a2da None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:32.932005 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dad5d2-7b4b-4ce0-ac6a-de570e2643a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:32.932605 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dad5d2-7b4b-4ce0-ac6a-de570e2643a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:32.932881 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dad5d2-7b4b-4ce0-ac6a-de570e2643a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:32.933152 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dad5d2-7b4b-4ce0-ac6a-de570e2643a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:32.933829 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dad5d2-7b4b-4ce0-ac6a-de570e2643a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:32.936679 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dad5d2-7b4b-4ce0-ac6a-de570e2643a1 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:33.024668 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1114/2238] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:40:32 2022] POST /identity/v3/projects => generated 383 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 07:40:33.026398 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abae92f3-8a04-4823-98e6-400a087ac8d5 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:33.040896 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abae92f3-8a04-4823-98e6-400a087ac8d5 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:33.078401 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dad5d2-7b4b-4ce0-ac6a-de570e2643a1 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:33.082748 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-19dad5d2-7b4b-4ce0-ac6a-de570e2643a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:33.140956 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb596bc-c1d7-4738-85d8-02a83660620a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:33.141785 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb596bc-c1d7-4738-85d8-02a83660620a admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:33.142201 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb596bc-c1d7-4738-85d8-02a83660620a admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:33.142640 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb596bc-c1d7-4738-85d8-02a83660620a admin admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:33.143954 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb596bc-c1d7-4738-85d8-02a83660620a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:33.146510 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb596bc-c1d7-4738-85d8-02a83660620a admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:33.188913 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dad5d2-7b4b-4ce0-ac6a-de570e2643a1 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:33.197251 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716b111c-946a-4972-a5b6-c3309c56c36b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:33.197945 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716b111c-946a-4972-a5b6-c3309c56c36b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:33.198380 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716b111c-946a-4972-a5b6-c3309c56c36b admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:33.198664 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716b111c-946a-4972-a5b6-c3309c56c36b admin admin] PATH_INFO: `/v3/users/71857fabfb1249f0be41e512ed63cb2f` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:33.199851 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716b111c-946a-4972-a5b6-c3309c56c36b admin admin] RBAC: Authorizing `identity:delete_user(user_id=71857fabfb1249f0be41e512ed63cb2f)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:33.202682 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716b111c-946a-4972-a5b6-c3309c56c36b admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:33.265748 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1125/2239] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:40:33 2022] POST /identity/v3/users => generated 292 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:40:33.267635 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb596bc-c1d7-4738-85d8-02a83660620a admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:33.279174 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb596bc-c1d7-4738-85d8-02a83660620a admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:33.285222 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deee1ca0-06cd-4f65-939c-bdda69b912e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:33.287070 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deee1ca0-06cd-4f65-939c-bdda69b912e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:33.287070 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deee1ca0-06cd-4f65-939c-bdda69b912e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:33.287070 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deee1ca0-06cd-4f65-939c-bdda69b912e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:33.287980 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deee1ca0-06cd-4f65-939c-bdda69b912e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:33.289367 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deee1ca0-06cd-4f65-939c-bdda69b912e3 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:33.302508 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1126/2240] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:40:33 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:40:33.306803 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deee1ca0-06cd-4f65-939c-bdda69b912e3 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:33.318650 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deee1ca0-06cd-4f65-939c-bdda69b912e3 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:33.324229 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa4218d-8177-4443-a29e-f9b7dec0f00e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:33.325087 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa4218d-8177-4443-a29e-f9b7dec0f00e admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:33.325514 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa4218d-8177-4443-a29e-f9b7dec0f00e admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:33.325993 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa4218d-8177-4443-a29e-f9b7dec0f00e admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:33.327097 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa4218d-8177-4443-a29e-f9b7dec0f00e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:33.330035 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa4218d-8177-4443-a29e-f9b7dec0f00e admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:33.341615 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1127/2241] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:40:33 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 07:40:33.345507 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa4218d-8177-4443-a29e-f9b7dec0f00e admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:33.358896 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa4218d-8177-4443-a29e-f9b7dec0f00e admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:33.363848 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f91f97-8167-473f-a142-36a951cfd335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:33.365249 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f91f97-8167-473f-a142-36a951cfd335 admin admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:33.365647 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f91f97-8167-473f-a142-36a951cfd335 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:33.365997 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f91f97-8167-473f-a142-36a951cfd335 admin admin] PATH_INFO: `/v3/projects/4cd35f5433434a5b89ee749c11530852/users/d3a1415d1376409ea8a7b34f078c7b13/roles/053e12af84e943e699460a9d76c2072b` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:33.385386 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f91f97-8167-473f-a142-36a951cfd335 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cd35f5433434a5b89ee749c11530852, user_id=d3a1415d1376409ea8a7b34f078c7b13, role_id=053e12af84e943e699460a9d76c2072b)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:33.388729 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f91f97-8167-473f-a142-36a951cfd335 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:33.416475 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-716b111c-946a-4972-a5b6-c3309c56c36b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71857fabfb1249f0be41e512ed63cb2f'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:33.417269 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-716b111c-946a-4972-a5b6-c3309c56c36b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71857fabfb1249f0be41e512ed63cb2f'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:33.470376 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1128/2242] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:40:33 2022] PUT /identity/v3/projects/4cd35f5433434a5b89ee749c11530852/users/d3a1415d1376409ea8a7b34f078c7b13/roles/053e12af84e943e699460a9d76c2072b => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 07:40:33.482350 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acf8282-4d53-4c37-b0b7-74b6ad0c3225 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:33.482645 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acf8282-4d53-4c37-b0b7-74b6ad0c3225 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:33.484262 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acf8282-4d53-4c37-b0b7-74b6ad0c3225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:33.518398 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-9acf8282-4d53-4c37-b0b7-74b6ad0c3225 None None] MFA Rules not processed for user `d3a1415d1376409ea8a7b34f078c7b13`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:40:33.520038 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-9acf8282-4d53-4c37-b0b7-74b6ad0c3225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:33.527934 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-716b111c-946a-4972-a5b6-c3309c56c36b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71857fabfb1249f0be41e512ed63cb2f'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:33.528657 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-716b111c-946a-4972-a5b6-c3309c56c36b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71857fabfb1249f0be41e512ed63cb2f'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:33.538272 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-716b111c-946a-4972-a5b6-c3309c56c36b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71857fabfb1249f0be41e512ed63cb2f'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:33.542697 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:40:33.561931 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1115/2243] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:40:33 2022] DELETE /identity/v3/users/71857fabfb1249f0be41e512ed63cb2f => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:40:33.566353 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd1ab64-0bb0-49d9-846a-87cc4b83bb5d None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:33.567279 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd1ab64-0bb0-49d9-846a-87cc4b83bb5d None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:33.567279 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd1ab64-0bb0-49d9-846a-87cc4b83bb5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:33.603338 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-dfd1ab64-0bb0-49d9-846a-87cc4b83bb5d None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:40:33.603338 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-dfd1ab64-0bb0-49d9-846a-87cc4b83bb5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:33.638248 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1129/2244] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:40:33 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:40:33.642839 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acf8282-4d53-4c37-b0b7-74b6ad0c3225 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:33.644661 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-9acf8282-4d53-4c37-b0b7-74b6ad0c3225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:33.678160 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:40:33.688683 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1116/2245] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:40:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 07:40:33.699214 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd1ab64-0bb0-49d9-846a-87cc4b83bb5d None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:33.701138 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-dfd1ab64-0bb0-49d9-846a-87cc4b83bb5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:33.721919 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acf8282-4d53-4c37-b0b7-74b6ad0c3225 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:33.726787 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849c4d43-421c-4308-aa98-549047eabafd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:33.727489 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849c4d43-421c-4308-aa98-549047eabafd admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:33.727774 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849c4d43-421c-4308-aa98-549047eabafd admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:33.728127 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849c4d43-421c-4308-aa98-549047eabafd admin admin] PATH_INFO: `/v3/projects` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:33.728777 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849c4d43-421c-4308-aa98-549047eabafd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:33.730807 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849c4d43-421c-4308-aa98-549047eabafd admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:33.762180 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd1ab64-0bb0-49d9-846a-87cc4b83bb5d None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:33.766735 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b5dfe5-989a-43f7-8dbf-b8057a23619d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:33.767358 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b5dfe5-989a-43f7-8dbf-b8057a23619d service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:33.767638 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b5dfe5-989a-43f7-8dbf-b8057a23619d service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:33.767951 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b5dfe5-989a-43f7-8dbf-b8057a23619d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:33.779914 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b5dfe5-989a-43f7-8dbf-b8057a23619d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:33.781373 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b5dfe5-989a-43f7-8dbf-b8057a23619d service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:33.791306 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1117/2246] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:40:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 07:40:33.852546 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1130/2247] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:40:33 2022] POST /identity/v3/projects => generated 381 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:40:33.856659 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b5dfe5-989a-43f7-8dbf-b8057a23619d service neutron] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:33.867986 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b5dfe5-989a-43f7-8dbf-b8057a23619d service neutron] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:33.933961 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbe8c63-a525-4e21-8f34-a61475e3dde8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:33.934686 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbe8c63-a525-4e21-8f34-a61475e3dde8 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:33.935034 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbe8c63-a525-4e21-8f34-a61475e3dde8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:33.935376 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbe8c63-a525-4e21-8f34-a61475e3dde8 admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:33.936655 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbe8c63-a525-4e21-8f34-a61475e3dde8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:33.938776 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbe8c63-a525-4e21-8f34-a61475e3dde8 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:34.035023 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1118/2248] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:40:33 2022] POST /identity/v3/users => generated 297 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:40:34.038858 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849c4d43-421c-4308-aa98-549047eabafd admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:34.049806 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849c4d43-421c-4308-aa98-549047eabafd admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:34.054414 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae4484e-1ba7-43dd-b41d-f61e8eab5102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:34.055012 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae4484e-1ba7-43dd-b41d-f61e8eab5102 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:34.055293 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae4484e-1ba7-43dd-b41d-f61e8eab5102 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:34.055569 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae4484e-1ba7-43dd-b41d-f61e8eab5102 admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:34.056208 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae4484e-1ba7-43dd-b41d-f61e8eab5102 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:34.057839 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae4484e-1ba7-43dd-b41d-f61e8eab5102 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:34.068487 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1131/2249] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:40:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:40:34.073128 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbe8c63-a525-4e21-8f34-a61475e3dde8 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:34.084270 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbe8c63-a525-4e21-8f34-a61475e3dde8 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:34.090263 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0148230e-2fba-4b0c-a56b-a56882f7ca79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:34.091189 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0148230e-2fba-4b0c-a56b-a56882f7ca79 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:34.091551 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0148230e-2fba-4b0c-a56b-a56882f7ca79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:34.092001 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0148230e-2fba-4b0c-a56b-a56882f7ca79 admin admin] PATH_INFO: `/v3/projects/e47703e65dc84f748c4e98539b644963/users/b32c9ebab3ba4e8b97f3b1d8e6ca67d6/roles/6bbb643e0d8d4bc0a9b27c26edf59216` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:34.114683 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0148230e-2fba-4b0c-a56b-a56882f7ca79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e47703e65dc84f748c4e98539b644963, user_id=b32c9ebab3ba4e8b97f3b1d8e6ca67d6, role_id=6bbb643e0d8d4bc0a9b27c26edf59216)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:34.118282 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0148230e-2fba-4b0c-a56b-a56882f7ca79 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:34.211745 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1119/2250] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:40:34 2022] PUT /identity/v3/projects/e47703e65dc84f748c4e98539b644963/users/b32c9ebab3ba4e8b97f3b1d8e6ca67d6/roles/6bbb643e0d8d4bc0a9b27c26edf59216 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:40:34.215627 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae4484e-1ba7-43dd-b41d-f61e8eab5102 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:34.235516 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae4484e-1ba7-43dd-b41d-f61e8eab5102 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:34.305093 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19350159-0032-4eab-949b-b220f6cf7469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:34.305835 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19350159-0032-4eab-949b-b220f6cf7469 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:34.306117 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19350159-0032-4eab-949b-b220f6cf7469 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:34.306387 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19350159-0032-4eab-949b-b220f6cf7469 admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:34.307028 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19350159-0032-4eab-949b-b220f6cf7469 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:34.308623 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19350159-0032-4eab-949b-b220f6cf7469 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:34.319313 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1132/2251] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:40:34 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:40:34.323931 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0148230e-2fba-4b0c-a56b-a56882f7ca79 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:34.335932 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0148230e-2fba-4b0c-a56b-a56882f7ca79 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:34.341022 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88db2e20-14eb-4d6a-8e6a-5a0b0ab26cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:34.341734 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88db2e20-14eb-4d6a-8e6a-5a0b0ab26cec admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:34.342922 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88db2e20-14eb-4d6a-8e6a-5a0b0ab26cec admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:34.342922 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88db2e20-14eb-4d6a-8e6a-5a0b0ab26cec admin admin] PATH_INFO: `/v3/projects/e47703e65dc84f748c4e98539b644963/users/b32c9ebab3ba4e8b97f3b1d8e6ca67d6/roles/6bbb643e0d8d4bc0a9b27c26edf59216` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:34.344429 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88db2e20-14eb-4d6a-8e6a-5a0b0ab26cec admin admin] RBAC: Authorizing `identity:create_grant(project_id=e47703e65dc84f748c4e98539b644963, user_id=b32c9ebab3ba4e8b97f3b1d8e6ca67d6, role_id=6bbb643e0d8d4bc0a9b27c26edf59216)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:34.347126 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88db2e20-14eb-4d6a-8e6a-5a0b0ab26cec admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:34.359054 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1120/2252] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:40:34 2022] PUT /identity/v3/projects/e47703e65dc84f748c4e98539b644963/users/b32c9ebab3ba4e8b97f3b1d8e6ca67d6/roles/6bbb643e0d8d4bc0a9b27c26edf59216 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:40:34.371403 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ff7b96-5b4d-400e-952b-bb1ead739d92 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:34.371961 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ff7b96-5b4d-400e-952b-bb1ead739d92 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:34.372683 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ff7b96-5b4d-400e-952b-bb1ead739d92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:34.414558 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-96ff7b96-5b4d-400e-952b-bb1ead739d92 None None] MFA Rules not processed for user `b32c9ebab3ba4e8b97f3b1d8e6ca67d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:40:34.415989 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-96ff7b96-5b4d-400e-952b-bb1ead739d92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:34.472006 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:40:34.542969 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1133/2253] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:40:34 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:40:34.555580 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88db2e20-14eb-4d6a-8e6a-5a0b0ab26cec admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:34.557357 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-88db2e20-14eb-4d6a-8e6a-5a0b0ab26cec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:34.630115 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ff7b96-5b4d-400e-952b-bb1ead739d92 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:34.632934 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-96ff7b96-5b4d-400e-952b-bb1ead739d92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:34.642698 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88db2e20-14eb-4d6a-8e6a-5a0b0ab26cec admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:34.647077 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676be12c-73f0-4b69-9b51-8f74bf627485 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecf6642b7b84d3c9756bb41ab3349d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:34.647419 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676be12c-73f0-4b69-9b51-8f74bf627485 service cinder] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:34.647666 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676be12c-73f0-4b69-9b51-8f74bf627485 service cinder] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:34.648118 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676be12c-73f0-4b69-9b51-8f74bf627485 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:34.661117 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676be12c-73f0-4b69-9b51-8f74bf627485 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:34.665153 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:40:34.666192 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676be12c-73f0-4b69-9b51-8f74bf627485 service cinder] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:34.690237 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1121/2254] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:40:34 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:40:34.781076 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ff7b96-5b4d-400e-952b-bb1ead739d92 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:34.786301 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04761c0-3e11-414a-aa54-60cb4a1116b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:34.786997 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04761c0-3e11-414a-aa54-60cb4a1116b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:34.787347 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04761c0-3e11-414a-aa54-60cb4a1116b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:34.787609 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04761c0-3e11-414a-aa54-60cb4a1116b2 admin admin] PATH_INFO: `/v3/projects/06390074fe824d749abd5ebf04604e48` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:34.788780 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04761c0-3e11-414a-aa54-60cb4a1116b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06390074fe824d749abd5ebf04604e48)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:34.790846 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04761c0-3e11-414a-aa54-60cb4a1116b2 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:34.897262 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676be12c-73f0-4b69-9b51-8f74bf627485 service cinder] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:34.908604 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676be12c-73f0-4b69-9b51-8f74bf627485 service cinder] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:34.914420 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a93e33a-69e1-4309-8ab9-ff8bc363ff06 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecf6642b7b84d3c9756bb41ab3349d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:34.915050 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a93e33a-69e1-4309-8ab9-ff8bc363ff06 service cinder] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:34.915319 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a93e33a-69e1-4309-8ab9-ff8bc363ff06 service cinder] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:34.915590 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a93e33a-69e1-4309-8ab9-ff8bc363ff06 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:34.933540 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-b04761c0-3e11-414a-aa54-60cb4a1116b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06390074fe824d749abd5ebf04604e48', 'request_id': 'req-b04761c0-3e11-414a-aa54-60cb4a1116b2'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:34.957523 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-b04761c0-3e11-414a-aa54-60cb4a1116b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06390074fe824d749abd5ebf04604e48', 'request_id': 'req-b04761c0-3e11-414a-aa54-60cb4a1116b2'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:35.033979 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a93e33a-69e1-4309-8ab9-ff8bc363ff06 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:35.035608 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a93e33a-69e1-4309-8ab9-ff8bc363ff06 service cinder] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:35.046572 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1122/2255] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:40:34 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 07:40:35.059019 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-b04761c0-3e11-414a-aa54-60cb4a1116b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06390074fe824d749abd5ebf04604e48', 'request_id': 'req-b04761c0-3e11-414a-aa54-60cb4a1116b2'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:35.080836 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-b04761c0-3e11-414a-aa54-60cb4a1116b2 admin admin] The token cache is being invalidate because project 06390074fe824d749abd5ebf04604e48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:40:35.080836 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-b04761c0-3e11-414a-aa54-60cb4a1116b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:35.081011 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-b04761c0-3e11-414a-aa54-60cb4a1116b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:35.082768 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1134/2256] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:40:34 2022] DELETE /identity/v3/projects/06390074fe824d749abd5ebf04604e48 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 07:40:35.087789 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe78c293-1009-4b21-80dc-396ab75e08e2 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:35.094525 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe78c293-1009-4b21-80dc-396ab75e08e2 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:35.098318 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe78c293-1009-4b21-80dc-396ab75e08e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:35.118424 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-fe78c293-1009-4b21-80dc-396ab75e08e2 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:40:35.119658 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-fe78c293-1009-4b21-80dc-396ab75e08e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:35.200467 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1123/2257] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:40:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:40:35.208873 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be1b943-c0af-433d-bfd4-6ec242280115 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:35.209198 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be1b943-c0af-433d-bfd4-6ec242280115 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:35.209483 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be1b943-c0af-433d-bfd4-6ec242280115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:35.231856 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-7be1b943-c0af-433d-bfd4-6ec242280115 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:40:35.233270 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-7be1b943-c0af-433d-bfd4-6ec242280115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:35.252270 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1135/2258] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:40:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:40:35.256041 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe78c293-1009-4b21-80dc-396ab75e08e2 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:35.270314 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe78c293-1009-4b21-80dc-396ab75e08e2 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:35.289210 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ae8519-5f72-4e62-ab6a-0a86536a51b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:35.289883 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ae8519-5f72-4e62-ab6a-0a86536a51b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:35.290166 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ae8519-5f72-4e62-ab6a-0a86536a51b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:35.290858 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ae8519-5f72-4e62-ab6a-0a86536a51b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:35.291442 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ae8519-5f72-4e62-ab6a-0a86536a51b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:40:35.291750 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ae8519-5f72-4e62-ab6a-0a86536a51b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:35.293394 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ae8519-5f72-4e62-ab6a-0a86536a51b5 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:35.309740 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1124/2259] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 15 07:40:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:40:35.314798 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be1b943-c0af-433d-bfd4-6ec242280115 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:35.331141 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be1b943-c0af-433d-bfd4-6ec242280115 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:35.338301 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c33b3fa-10ad-4fb3-a83a-8d0139b85a62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:35.338601 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c33b3fa-10ad-4fb3-a83a-8d0139b85a62 admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:35.339091 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c33b3fa-10ad-4fb3-a83a-8d0139b85a62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:35.339579 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c33b3fa-10ad-4fb3-a83a-8d0139b85a62 admin admin] PATH_INFO: `/v3/projects` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:35.340369 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c33b3fa-10ad-4fb3-a83a-8d0139b85a62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:35.343255 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c33b3fa-10ad-4fb3-a83a-8d0139b85a62 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:35.462583 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1136/2260] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:40:35 2022] POST /identity/v3/projects => generated 371 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:40:35.464017 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ae8519-5f72-4e62-ab6a-0a86536a51b5 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:35.492084 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ae8519-5f72-4e62-ab6a-0a86536a51b5 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:35.564711 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89cba33-9ee4-4329-9a7a-0fe729de407f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:35.565365 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89cba33-9ee4-4329-9a7a-0fe729de407f admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:35.565623 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89cba33-9ee4-4329-9a7a-0fe729de407f admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:35.565921 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89cba33-9ee4-4329-9a7a-0fe729de407f admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:35.566550 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89cba33-9ee4-4329-9a7a-0fe729de407f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:35.568500 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89cba33-9ee4-4329-9a7a-0fe729de407f admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:35.651581 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c33b3fa-10ad-4fb3-a83a-8d0139b85a62 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:35.653445 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-9c33b3fa-10ad-4fb3-a83a-8d0139b85a62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:35.669363 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c33b3fa-10ad-4fb3-a83a-8d0139b85a62 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:35.674363 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e6a1cb-eb8e-4f93-9002-274cd8a32389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:35.674999 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e6a1cb-eb8e-4f93-9002-274cd8a32389 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:35.675279 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e6a1cb-eb8e-4f93-9002-274cd8a32389 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:35.675557 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e6a1cb-eb8e-4f93-9002-274cd8a32389 admin admin] PATH_INFO: `/v3/users/d3a1415d1376409ea8a7b34f078c7b13` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:35.676701 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e6a1cb-eb8e-4f93-9002-274cd8a32389 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3a1415d1376409ea8a7b34f078c7b13)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:35.678661 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e6a1cb-eb8e-4f93-9002-274cd8a32389 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:35.694755 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:40:35.700648 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1125/2261] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:40:35 2022] POST /identity/v3/users => generated 286 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 07:40:35.704545 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89cba33-9ee4-4329-9a7a-0fe729de407f admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:35.716649 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89cba33-9ee4-4329-9a7a-0fe729de407f admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:35.729435 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63279b6-34a8-4355-91ac-7a637c9f94fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:35.729435 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63279b6-34a8-4355-91ac-7a637c9f94fa admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:35.729435 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63279b6-34a8-4355-91ac-7a637c9f94fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:35.729435 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63279b6-34a8-4355-91ac-7a637c9f94fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:35.730126 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63279b6-34a8-4355-91ac-7a637c9f94fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:35.731741 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63279b6-34a8-4355-91ac-7a637c9f94fa admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:35.746875 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:40:35.755093 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1126/2262] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:40:35 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:40:35.760381 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63279b6-34a8-4355-91ac-7a637c9f94fa admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:35.777503 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63279b6-34a8-4355-91ac-7a637c9f94fa admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:35.785954 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f71866e-74f4-4d14-a7d1-9823df792ae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:35.786740 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f71866e-74f4-4d14-a7d1-9823df792ae6 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:35.787133 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f71866e-74f4-4d14-a7d1-9823df792ae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:35.787477 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f71866e-74f4-4d14-a7d1-9823df792ae6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:35.788200 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f71866e-74f4-4d14-a7d1-9823df792ae6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:35.790459 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f71866e-74f4-4d14-a7d1-9823df792ae6 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:35.809643 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1127/2263] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:40:35 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:40:35.813916 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f71866e-74f4-4d14-a7d1-9823df792ae6 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:35.823774 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f71866e-74f4-4d14-a7d1-9823df792ae6 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:35.829546 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53d350c-5f48-48c3-a066-92244eacdfe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:35.830270 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53d350c-5f48-48c3-a066-92244eacdfe5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:35.830522 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53d350c-5f48-48c3-a066-92244eacdfe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:35.830795 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53d350c-5f48-48c3-a066-92244eacdfe5 admin admin] PATH_INFO: `/v3/projects/a53d9cbe31d5495c9bb78d0e7fd5522e/users/ce0a1d3c36cb46a6947983b223e20b1b/roles/053e12af84e943e699460a9d76c2072b` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:35.850452 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53d350c-5f48-48c3-a066-92244eacdfe5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a53d9cbe31d5495c9bb78d0e7fd5522e, user_id=ce0a1d3c36cb46a6947983b223e20b1b, role_id=053e12af84e943e699460a9d76c2072b)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:35.855509 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53d350c-5f48-48c3-a066-92244eacdfe5 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:35.915035 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-80e6a1cb-eb8e-4f93-9002-274cd8a32389 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3a1415d1376409ea8a7b34f078c7b13'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:35.970049 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1128/2264] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:40:35 2022] PUT /identity/v3/projects/a53d9cbe31d5495c9bb78d0e7fd5522e/users/ce0a1d3c36cb46a6947983b223e20b1b/roles/053e12af84e943e699460a9d76c2072b => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:40:35.977342 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c3e6e4-5759-4510-8d0a-671ff3e370dd None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:35.977658 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c3e6e4-5759-4510-8d0a-671ff3e370dd None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:35.977928 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c3e6e4-5759-4510-8d0a-671ff3e370dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:35.987890 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-80e6a1cb-eb8e-4f93-9002-274cd8a32389 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3a1415d1376409ea8a7b34f078c7b13'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:35.988653 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-80e6a1cb-eb8e-4f93-9002-274cd8a32389 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3a1415d1376409ea8a7b34f078c7b13'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:35.989366 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-80e6a1cb-eb8e-4f93-9002-274cd8a32389 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3a1415d1376409ea8a7b34f078c7b13'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:36.001289 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-80e6a1cb-eb8e-4f93-9002-274cd8a32389 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3a1415d1376409ea8a7b34f078c7b13'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:36.027967 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-e3c3e6e4-5759-4510-8d0a-671ff3e370dd None None] MFA Rules not processed for user `ce0a1d3c36cb46a6947983b223e20b1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:40:36.029240 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-e3c3e6e4-5759-4510-8d0a-671ff3e370dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:36.038073 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1137/2265] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:40:35 2022] DELETE /identity/v3/users/d3a1415d1376409ea8a7b34f078c7b13 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 07:40:36.042525 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2804dbc-cbaa-40bc-9a19-5335b2f02871 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:36.043005 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2804dbc-cbaa-40bc-9a19-5335b2f02871 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:36.043439 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2804dbc-cbaa-40bc-9a19-5335b2f02871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:36.066067 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-c2804dbc-cbaa-40bc-9a19-5335b2f02871 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:40:36.067732 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-c2804dbc-cbaa-40bc-9a19-5335b2f02871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:36.143318 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1129/2266] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:40:35 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:40:36.149802 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1138/2267] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:40:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 07:40:36.157699 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c3e6e4-5759-4510-8d0a-671ff3e370dd None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:36.159116 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-e3c3e6e4-5759-4510-8d0a-671ff3e370dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:36.164431 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2804dbc-cbaa-40bc-9a19-5335b2f02871 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:36.166298 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-c2804dbc-cbaa-40bc-9a19-5335b2f02871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:36.230896 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2804dbc-cbaa-40bc-9a19-5335b2f02871 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:36.231384 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c3e6e4-5759-4510-8d0a-671ff3e370dd None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:36.235079 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093d26fd-70f1-4a2e-9c39-377fd7c7aae3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:36.235916 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093d26fd-70f1-4a2e-9c39-377fd7c7aae3 service neutron] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:36.236197 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093d26fd-70f1-4a2e-9c39-377fd7c7aae3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:36.236469 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093d26fd-70f1-4a2e-9c39-377fd7c7aae3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:36.250848 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093d26fd-70f1-4a2e-9c39-377fd7c7aae3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:36.252767 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093d26fd-70f1-4a2e-9c39-377fd7c7aae3 service neutron] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:36.267830 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1139/2268] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:40:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 07:40:36.289878 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71883b1-40ee-494a-b780-91ef2f9ac3cf service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37438b032a1f46d3bdf7121ad0cffbc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:36.290704 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71883b1-40ee-494a-b780-91ef2f9ac3cf service glance] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:36.291202 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71883b1-40ee-494a-b780-91ef2f9ac3cf service glance] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:36.291550 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71883b1-40ee-494a-b780-91ef2f9ac3cf service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:36.358720 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71883b1-40ee-494a-b780-91ef2f9ac3cf service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:36.360170 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71883b1-40ee-494a-b780-91ef2f9ac3cf service glance] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:36.370214 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1130/2269] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:40:36 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 07:40:36.459386 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:40:37.177452 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093d26fd-70f1-4a2e-9c39-377fd7c7aae3 service neutron] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:37.179062 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-093d26fd-70f1-4a2e-9c39-377fd7c7aae3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:37.193446 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093d26fd-70f1-4a2e-9c39-377fd7c7aae3 service neutron] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:37.198741 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffbd958-a7bd-486e-b546-88af2f690d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:37.199376 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffbd958-a7bd-486e-b546-88af2f690d05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:37.199654 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffbd958-a7bd-486e-b546-88af2f690d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:37.199929 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffbd958-a7bd-486e-b546-88af2f690d05 admin admin] PATH_INFO: `/v3/projects/4cd35f5433434a5b89ee749c11530852` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:37.201072 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffbd958-a7bd-486e-b546-88af2f690d05 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4cd35f5433434a5b89ee749c11530852)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:37.203299 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffbd958-a7bd-486e-b546-88af2f690d05 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:37.295715 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:40:37.339248 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-9ffbd958-a7bd-486e-b546-88af2f690d05 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cd35f5433434a5b89ee749c11530852', 'request_id': 'req-9ffbd958-a7bd-486e-b546-88af2f690d05'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:37.348048 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-9ffbd958-a7bd-486e-b546-88af2f690d05 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cd35f5433434a5b89ee749c11530852', 'request_id': 'req-9ffbd958-a7bd-486e-b546-88af2f690d05'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:37.442213 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-9ffbd958-a7bd-486e-b546-88af2f690d05 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cd35f5433434a5b89ee749c11530852', 'request_id': 'req-9ffbd958-a7bd-486e-b546-88af2f690d05'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:37.454333 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-9ffbd958-a7bd-486e-b546-88af2f690d05 admin admin] The token cache is being invalidate because project 4cd35f5433434a5b89ee749c11530852 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:40:37.454633 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-9ffbd958-a7bd-486e-b546-88af2f690d05 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:37.455331 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-9ffbd958-a7bd-486e-b546-88af2f690d05 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:37.456797 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1140/2270] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:40:37 2022] DELETE /identity/v3/projects/4cd35f5433434a5b89ee749c11530852 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:40:37.461123 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71883b1-40ee-494a-b780-91ef2f9ac3cf service glance] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:37.472360 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71883b1-40ee-494a-b780-91ef2f9ac3cf service glance] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:37.536903 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acfe4d4-5bcc-4155-bcc1-359a3b2b851c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:37.537538 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acfe4d4-5bcc-4155-bcc1-359a3b2b851c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:37.537858 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acfe4d4-5bcc-4155-bcc1-359a3b2b851c admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:37.538131 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acfe4d4-5bcc-4155-bcc1-359a3b2b851c admin admin] PATH_INFO: `/v3/users/b32c9ebab3ba4e8b97f3b1d8e6ca67d6` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:37.539240 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acfe4d4-5bcc-4155-bcc1-359a3b2b851c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b32c9ebab3ba4e8b97f3b1d8e6ca67d6)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:37.541199 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acfe4d4-5bcc-4155-bcc1-359a3b2b851c admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:37.716014 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-5acfe4d4-5bcc-4155-bcc1-359a3b2b851c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b32c9ebab3ba4e8b97f3b1d8e6ca67d6'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:37.734135 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-5acfe4d4-5bcc-4155-bcc1-359a3b2b851c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b32c9ebab3ba4e8b97f3b1d8e6ca67d6'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:37.827657 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-5acfe4d4-5bcc-4155-bcc1-359a3b2b851c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b32c9ebab3ba4e8b97f3b1d8e6ca67d6'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:37.832972 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-5acfe4d4-5bcc-4155-bcc1-359a3b2b851c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b32c9ebab3ba4e8b97f3b1d8e6ca67d6'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:37.838332 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-5acfe4d4-5bcc-4155-bcc1-359a3b2b851c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b32c9ebab3ba4e8b97f3b1d8e6ca67d6'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:37.862209 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1131/2271] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:40:37 2022] DELETE /identity/v3/users/b32c9ebab3ba4e8b97f3b1d8e6ca67d6 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:40:38.037707 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd039cc2-d2ed-46a5-af33-1e049590579e None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:38.038001 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd039cc2-d2ed-46a5-af33-1e049590579e None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:38.038279 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd039cc2-d2ed-46a5-af33-1e049590579e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:38.057522 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-fd039cc2-d2ed-46a5-af33-1e049590579e None None] MFA Rules not processed for user `5e0c8d7aa4db455a94631becf212866f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:40:38.058771 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-fd039cc2-d2ed-46a5-af33-1e049590579e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:38.117490 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1141/2272] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:40:38 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:40:38.567140 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acfe4d4-5bcc-4155-bcc1-359a3b2b851c admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:38.568569 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-5acfe4d4-5bcc-4155-bcc1-359a3b2b851c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:38.638050 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:40:38.658679 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acfe4d4-5bcc-4155-bcc1-359a3b2b851c admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:38.663460 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db720758-e8db-4ba6-ab13-72580fa35afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:38.664146 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db720758-e8db-4ba6-ab13-72580fa35afa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:38.664426 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db720758-e8db-4ba6-ab13-72580fa35afa admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:38.664699 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db720758-e8db-4ba6-ab13-72580fa35afa admin admin] PATH_INFO: `/v3/projects/e47703e65dc84f748c4e98539b644963` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:38.666371 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db720758-e8db-4ba6-ab13-72580fa35afa admin admin] RBAC: Authorizing `identity:delete_project(project_id=e47703e65dc84f748c4e98539b644963)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:38.668231 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db720758-e8db-4ba6-ab13-72580fa35afa admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:38.777521 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-db720758-e8db-4ba6-ab13-72580fa35afa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e47703e65dc84f748c4e98539b644963', 'request_id': 'req-db720758-e8db-4ba6-ab13-72580fa35afa'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:38.785963 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-db720758-e8db-4ba6-ab13-72580fa35afa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e47703e65dc84f748c4e98539b644963', 'request_id': 'req-db720758-e8db-4ba6-ab13-72580fa35afa'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:38.861839 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-db720758-e8db-4ba6-ab13-72580fa35afa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e47703e65dc84f748c4e98539b644963', 'request_id': 'req-db720758-e8db-4ba6-ab13-72580fa35afa'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:38.873073 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-db720758-e8db-4ba6-ab13-72580fa35afa admin admin] The token cache is being invalidate because project e47703e65dc84f748c4e98539b644963 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:40:38.873373 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-db720758-e8db-4ba6-ab13-72580fa35afa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:38.874531 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-db720758-e8db-4ba6-ab13-72580fa35afa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:38.876060 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1132/2273] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:40:38 2022] DELETE /identity/v3/projects/e47703e65dc84f748c4e98539b644963 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:40:38.881242 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706d8835-4738-4e73-911c-2eee261d03e5 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:38.881822 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706d8835-4738-4e73-911c-2eee261d03e5 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:38.881942 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706d8835-4738-4e73-911c-2eee261d03e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:38.903140 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-706d8835-4738-4e73-911c-2eee261d03e5 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:40:38.904552 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-706d8835-4738-4e73-911c-2eee261d03e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:39.001545 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1142/2274] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:40:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:40:39.009475 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e8487a-04b4-49ee-9c0f-c6c57af7d7c8 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:39.009860 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e8487a-04b4-49ee-9c0f-c6c57af7d7c8 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:39.010242 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e8487a-04b4-49ee-9c0f-c6c57af7d7c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:39.032423 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-c5e8487a-04b4-49ee-9c0f-c6c57af7d7c8 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:40:39.033803 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-c5e8487a-04b4-49ee-9c0f-c6c57af7d7c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:39.045395 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1133/2275] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:40:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:40:39.046892 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706d8835-4738-4e73-911c-2eee261d03e5 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:39.057092 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706d8835-4738-4e73-911c-2eee261d03e5 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:39.061814 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9373e99-e8d2-4436-acd1-e0b1173dc7a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:39.062441 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9373e99-e8d2-4436-acd1-e0b1173dc7a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:39.062719 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9373e99-e8d2-4436-acd1-e0b1173dc7a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:39.062993 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9373e99-e8d2-4436-acd1-e0b1173dc7a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:39.063645 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9373e99-e8d2-4436-acd1-e0b1173dc7a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:40:39.063952 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9373e99-e8d2-4436-acd1-e0b1173dc7a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:39.065611 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9373e99-e8d2-4436-acd1-e0b1173dc7a7 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:39.082534 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1143/2276] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 15 07:40:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:40:39.083392 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e8487a-04b4-49ee-9c0f-c6c57af7d7c8 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:39.094112 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e8487a-04b4-49ee-9c0f-c6c57af7d7c8 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:39.098795 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f36196d-76b8-4714-b9ba-392556eaed03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:39.099405 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f36196d-76b8-4714-b9ba-392556eaed03 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:39.099685 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f36196d-76b8-4714-b9ba-392556eaed03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:39.099958 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f36196d-76b8-4714-b9ba-392556eaed03 admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:39.100587 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f36196d-76b8-4714-b9ba-392556eaed03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:39.102505 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f36196d-76b8-4714-b9ba-392556eaed03 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:39.206268 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1134/2277] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:40:39 2022] POST /identity/v3/projects => generated 371 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:40:39.210588 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9373e99-e8d2-4436-acd1-e0b1173dc7a7 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:39.222249 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9373e99-e8d2-4436-acd1-e0b1173dc7a7 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:39.270828 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:40:39.285788 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7b3496-f050-46cf-a434-565245cc721c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:39.286407 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7b3496-f050-46cf-a434-565245cc721c admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:39.286690 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7b3496-f050-46cf-a434-565245cc721c admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:39.286966 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7b3496-f050-46cf-a434-565245cc721c admin admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:39.287614 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7b3496-f050-46cf-a434-565245cc721c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:39.289890 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7b3496-f050-46cf-a434-565245cc721c admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:39.399325 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1144/2278] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:40:39 2022] POST /identity/v3/users => generated 286 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:40:39.403397 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f36196d-76b8-4714-b9ba-392556eaed03 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:39.414121 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f36196d-76b8-4714-b9ba-392556eaed03 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:39.419459 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13fa516-1992-4604-a82a-25fdbd0f1d5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:39.420070 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13fa516-1992-4604-a82a-25fdbd0f1d5b admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:39.420420 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13fa516-1992-4604-a82a-25fdbd0f1d5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:39.420661 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13fa516-1992-4604-a82a-25fdbd0f1d5b admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:39.421279 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13fa516-1992-4604-a82a-25fdbd0f1d5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:39.422919 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13fa516-1992-4604-a82a-25fdbd0f1d5b admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:39.452881 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1135/2279] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:40:39 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:40:39.458997 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7b3496-f050-46cf-a434-565245cc721c admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:39.469157 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7b3496-f050-46cf-a434-565245cc721c admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:39.474451 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6739511-7494-4213-8d42-c477cf17b119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:39.475855 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6739511-7494-4213-8d42-c477cf17b119 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:39.476402 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6739511-7494-4213-8d42-c477cf17b119 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:39.476829 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6739511-7494-4213-8d42-c477cf17b119 admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:39.478188 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6739511-7494-4213-8d42-c477cf17b119 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:39.479944 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6739511-7494-4213-8d42-c477cf17b119 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:39.491636 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1145/2280] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:40:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:40:39.496039 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13fa516-1992-4604-a82a-25fdbd0f1d5b admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:39.507522 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13fa516-1992-4604-a82a-25fdbd0f1d5b admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:39.512188 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e693a639-8163-40f7-8539-8609a8f80580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:39.512877 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e693a639-8163-40f7-8539-8609a8f80580 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:39.513158 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e693a639-8163-40f7-8539-8609a8f80580 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:39.513432 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e693a639-8163-40f7-8539-8609a8f80580 admin admin] PATH_INFO: `/v3/projects/8ba0fb62439b464aae93e173370bda0e/users/05dc0fa3bb084ca08270cbec6ddadc51/roles/053e12af84e943e699460a9d76c2072b` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:39.534225 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e693a639-8163-40f7-8539-8609a8f80580 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ba0fb62439b464aae93e173370bda0e, user_id=05dc0fa3bb084ca08270cbec6ddadc51, role_id=053e12af84e943e699460a9d76c2072b)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:39.537413 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e693a639-8163-40f7-8539-8609a8f80580 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:39.632812 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1136/2281] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:40:39 2022] PUT /identity/v3/projects/8ba0fb62439b464aae93e173370bda0e/users/05dc0fa3bb084ca08270cbec6ddadc51/roles/053e12af84e943e699460a9d76c2072b => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:40:39.645227 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d85f4e-2ad2-4058-a779-7d6ae2eed1c2 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:39.645540 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d85f4e-2ad2-4058-a779-7d6ae2eed1c2 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:39.645813 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d85f4e-2ad2-4058-a779-7d6ae2eed1c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:39.677009 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-66d85f4e-2ad2-4058-a779-7d6ae2eed1c2 None None] MFA Rules not processed for user `05dc0fa3bb084ca08270cbec6ddadc51`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:40:39.678308 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-66d85f4e-2ad2-4058-a779-7d6ae2eed1c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:39.761525 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1146/2282] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:40:39 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:40:39.772918 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e693a639-8163-40f7-8539-8609a8f80580 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:39.776126 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-e693a639-8163-40f7-8539-8609a8f80580 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:39.851593 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e693a639-8163-40f7-8539-8609a8f80580 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:39.855317 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2939ec6-9ec1-4187-af29-f2f64b051644 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecf6642b7b84d3c9756bb41ab3349d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:39.856023 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2939ec6-9ec1-4187-af29-f2f64b051644 service cinder] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:39.856309 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2939ec6-9ec1-4187-af29-f2f64b051644 service cinder] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:39.856584 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2939ec6-9ec1-4187-af29-f2f64b051644 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:39.866437 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2939ec6-9ec1-4187-af29-f2f64b051644 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:39.867866 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2939ec6-9ec1-4187-af29-f2f64b051644 service cinder] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:39.878041 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1137/2283] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:40:39 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:40:39.900799 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d85f4e-2ad2-4058-a779-7d6ae2eed1c2 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:39.991540 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d85f4e-2ad2-4058-a779-7d6ae2eed1c2 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:39.996342 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a50f6d7-a6cb-41f1-b435-a77cdd36964e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:39.996973 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a50f6d7-a6cb-41f1-b435-a77cdd36964e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:39.997260 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a50f6d7-a6cb-41f1-b435-a77cdd36964e admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:39.997530 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a50f6d7-a6cb-41f1-b435-a77cdd36964e admin admin] PATH_INFO: `/v3/users/05dc0fa3bb084ca08270cbec6ddadc51` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:39.998787 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a50f6d7-a6cb-41f1-b435-a77cdd36964e admin admin] RBAC: Authorizing `identity:delete_user(user_id=05dc0fa3bb084ca08270cbec6ddadc51)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:40.001592 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a50f6d7-a6cb-41f1-b435-a77cdd36964e admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:40.234693 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-7a50f6d7-a6cb-41f1-b435-a77cdd36964e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05dc0fa3bb084ca08270cbec6ddadc51'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:40.305410 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-7a50f6d7-a6cb-41f1-b435-a77cdd36964e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05dc0fa3bb084ca08270cbec6ddadc51'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:40.306043 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-7a50f6d7-a6cb-41f1-b435-a77cdd36964e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05dc0fa3bb084ca08270cbec6ddadc51'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:40.306658 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-7a50f6d7-a6cb-41f1-b435-a77cdd36964e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05dc0fa3bb084ca08270cbec6ddadc51'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:40.309107 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:40:40.316297 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-7a50f6d7-a6cb-41f1-b435-a77cdd36964e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05dc0fa3bb084ca08270cbec6ddadc51'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:40.335598 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1147/2284] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:40:39 2022] DELETE /identity/v3/users/05dc0fa3bb084ca08270cbec6ddadc51 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 07:40:40.346405 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffae514e-aa2d-4bfb-9824-ac85e5eded96 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:40.346720 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffae514e-aa2d-4bfb-9824-ac85e5eded96 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:40.347004 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffae514e-aa2d-4bfb-9824-ac85e5eded96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:40.368079 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-ffae514e-aa2d-4bfb-9824-ac85e5eded96 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:40:40.370627 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-ffae514e-aa2d-4bfb-9824-ac85e5eded96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:40.380992 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:40:40.435918 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1138/2285] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:40:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:40:40.449532 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a50f6d7-a6cb-41f1-b435-a77cdd36964e admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:40.449532 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-7a50f6d7-a6cb-41f1-b435-a77cdd36964e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:40.518861 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a50f6d7-a6cb-41f1-b435-a77cdd36964e admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:40.522706 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8c9421-c534-47aa-ade1-c6b9c92eef59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:40.523564 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c9421-c534-47aa-ade1-c6b9c92eef59 service neutron] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:40.523684 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c9421-c534-47aa-ade1-c6b9c92eef59 service neutron] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:40.523876 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c9421-c534-47aa-ade1-c6b9c92eef59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:40.534486 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8c9421-c534-47aa-ade1-c6b9c92eef59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:40.535945 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8c9421-c534-47aa-ade1-c6b9c92eef59 service neutron] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:40.549819 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1148/2286] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:40:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:40:41.032898 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffae514e-aa2d-4bfb-9824-ac85e5eded96 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:41.034783 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-ffae514e-aa2d-4bfb-9824-ac85e5eded96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:41.062249 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffae514e-aa2d-4bfb-9824-ac85e5eded96 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:41.070913 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6f4c51-32de-449b-931b-2506b6bb8c04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:41.071687 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6f4c51-32de-449b-931b-2506b6bb8c04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:41.072042 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6f4c51-32de-449b-931b-2506b6bb8c04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:41.072391 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6f4c51-32de-449b-931b-2506b6bb8c04 admin admin] PATH_INFO: `/v3/projects/8ba0fb62439b464aae93e173370bda0e` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:41.073875 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6f4c51-32de-449b-931b-2506b6bb8c04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ba0fb62439b464aae93e173370bda0e)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:41.075953 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6f4c51-32de-449b-931b-2506b6bb8c04 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:41.193146 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-2b6f4c51-32de-449b-931b-2506b6bb8c04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ba0fb62439b464aae93e173370bda0e', 'request_id': 'req-2b6f4c51-32de-449b-931b-2506b6bb8c04'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:41.199664 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-2b6f4c51-32de-449b-931b-2506b6bb8c04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ba0fb62439b464aae93e173370bda0e', 'request_id': 'req-2b6f4c51-32de-449b-931b-2506b6bb8c04'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:41.290769 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-2b6f4c51-32de-449b-931b-2506b6bb8c04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ba0fb62439b464aae93e173370bda0e', 'request_id': 'req-2b6f4c51-32de-449b-931b-2506b6bb8c04'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:41.302967 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-2b6f4c51-32de-449b-931b-2506b6bb8c04 admin admin] The token cache is being invalidate because project 8ba0fb62439b464aae93e173370bda0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:40:41.303264 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-2b6f4c51-32de-449b-931b-2506b6bb8c04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:41.303936 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-2b6f4c51-32de-449b-931b-2506b6bb8c04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:41.305420 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1139/2287] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:40:41 2022] DELETE /identity/v3/projects/8ba0fb62439b464aae93e173370bda0e => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:40:41.310377 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa1820f-6dca-4610-86bd-d8c984925c26 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:41.310857 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa1820f-6dca-4610-86bd-d8c984925c26 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:41.310947 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa1820f-6dca-4610-86bd-d8c984925c26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:41.331888 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-9fa1820f-6dca-4610-86bd-d8c984925c26 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:40:41.333229 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-9fa1820f-6dca-4610-86bd-d8c984925c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:41.413631 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1149/2288] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:40:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:40:41.422269 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48e957a-c8c0-4089-9fd0-5a7a49c44370 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:41.422269 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48e957a-c8c0-4089-9fd0-5a7a49c44370 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:41.422269 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48e957a-c8c0-4089-9fd0-5a7a49c44370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:41.442498 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-c48e957a-c8c0-4089-9fd0-5a7a49c44370 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:40:41.443511 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-c48e957a-c8c0-4089-9fd0-5a7a49c44370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:41.452497 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1140/2289] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:40:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:40:41.457348 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa1820f-6dca-4610-86bd-d8c984925c26 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:41.471135 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa1820f-6dca-4610-86bd-d8c984925c26 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:41.477315 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29daabb9-63fe-4919-b087-421c7f873a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:41.477979 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29daabb9-63fe-4919-b087-421c7f873a12 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:41.478262 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29daabb9-63fe-4919-b087-421c7f873a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:41.478547 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29daabb9-63fe-4919-b087-421c7f873a12 admin admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:41.479376 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29daabb9-63fe-4919-b087-421c7f873a12 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:40:41.479688 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29daabb9-63fe-4919-b087-421c7f873a12 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:41.481314 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29daabb9-63fe-4919-b087-421c7f873a12 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:41.500268 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1150/2290] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 15 07:40:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:40:41.504536 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48e957a-c8c0-4089-9fd0-5a7a49c44370 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:41.514924 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48e957a-c8c0-4089-9fd0-5a7a49c44370 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:41.519665 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5204f2ed-14d5-4f83-8f8f-3160fe27e652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:41.520260 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5204f2ed-14d5-4f83-8f8f-3160fe27e652 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:41.520562 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5204f2ed-14d5-4f83-8f8f-3160fe27e652 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:41.520821 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5204f2ed-14d5-4f83-8f8f-3160fe27e652 admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:41.521462 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5204f2ed-14d5-4f83-8f8f-3160fe27e652 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:41.523384 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5204f2ed-14d5-4f83-8f8f-3160fe27e652 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:41.652180 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1141/2291] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:40:41 2022] POST /identity/v3/projects => generated 373 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:40:41.657294 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29daabb9-63fe-4919-b087-421c7f873a12 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:41.670388 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29daabb9-63fe-4919-b087-421c7f873a12 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:41.739107 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2575f09a-82c4-4ed6-b969-a2f919899298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:41.739780 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2575f09a-82c4-4ed6-b969-a2f919899298 admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:41.740296 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2575f09a-82c4-4ed6-b969-a2f919899298 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:41.740604 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2575f09a-82c4-4ed6-b969-a2f919899298 admin admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:41.741250 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2575f09a-82c4-4ed6-b969-a2f919899298 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:41.743279 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2575f09a-82c4-4ed6-b969-a2f919899298 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:41.757247 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:40:41.831425 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1151/2292] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:40:41 2022] POST /identity/v3/users => generated 287 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 07:40:41.835749 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5204f2ed-14d5-4f83-8f8f-3160fe27e652 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:41.846680 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5204f2ed-14d5-4f83-8f8f-3160fe27e652 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:41.851982 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db595cb7-5a24-44b8-a43f-e92243ffe08a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:41.852637 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db595cb7-5a24-44b8-a43f-e92243ffe08a admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:41.852843 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db595cb7-5a24-44b8-a43f-e92243ffe08a admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:41.853114 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db595cb7-5a24-44b8-a43f-e92243ffe08a admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:41.853808 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db595cb7-5a24-44b8-a43f-e92243ffe08a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:41.855380 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db595cb7-5a24-44b8-a43f-e92243ffe08a admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:41.869007 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1142/2293] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:40:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 07:40:41.874857 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2575f09a-82c4-4ed6-b969-a2f919899298 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:41.887480 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2575f09a-82c4-4ed6-b969-a2f919899298 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:41.894930 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfac190-da20-4721-ab96-cf171f1dd3c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:41.895558 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfac190-da20-4721-ab96-cf171f1dd3c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:41.895840 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfac190-da20-4721-ab96-cf171f1dd3c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:41.896117 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfac190-da20-4721-ab96-cf171f1dd3c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:41.896807 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfac190-da20-4721-ab96-cf171f1dd3c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:41.898454 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfac190-da20-4721-ab96-cf171f1dd3c8 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:41.911709 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1152/2294] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:40:41 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:40:41.914174 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db595cb7-5a24-44b8-a43f-e92243ffe08a admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:41.927201 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db595cb7-5a24-44b8-a43f-e92243ffe08a admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:41.935186 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a6c277-b596-46af-adea-a60e9a971e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:41.935860 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a6c277-b596-46af-adea-a60e9a971e6e admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:41.936139 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a6c277-b596-46af-adea-a60e9a971e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:41.937052 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a6c277-b596-46af-adea-a60e9a971e6e admin admin] PATH_INFO: `/v3/projects/4ae2b75b233d499b8f9213b6a061c159/users/ffc376ff160d45e4964c34d59ade6c5b/roles/053e12af84e943e699460a9d76c2072b` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:41.962613 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a6c277-b596-46af-adea-a60e9a971e6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ae2b75b233d499b8f9213b6a061c159, user_id=ffc376ff160d45e4964c34d59ade6c5b, role_id=053e12af84e943e699460a9d76c2072b)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:41.965607 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a6c277-b596-46af-adea-a60e9a971e6e admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:42.064816 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1143/2295] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:40:41 2022] PUT /identity/v3/projects/4ae2b75b233d499b8f9213b6a061c159/users/ffc376ff160d45e4964c34d59ade6c5b/roles/053e12af84e943e699460a9d76c2072b => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:40:42.081000 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45744e62-7928-4a1e-a0ee-8daec02a1439 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:42.081269 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45744e62-7928-4a1e-a0ee-8daec02a1439 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:42.081550 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45744e62-7928-4a1e-a0ee-8daec02a1439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:42.140945 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-45744e62-7928-4a1e-a0ee-8daec02a1439 None None] MFA Rules not processed for user `ffc376ff160d45e4964c34d59ade6c5b`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:40:42.142538 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-45744e62-7928-4a1e-a0ee-8daec02a1439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:42.292648 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1153/2296] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:40:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 07:40:42.306386 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a6c277-b596-46af-adea-a60e9a971e6e admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:42.308853 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-61a6c277-b596-46af-adea-a60e9a971e6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:42.376931 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:40:42.415319 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a6c277-b596-46af-adea-a60e9a971e6e admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:42.419834 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0dabda-f9fc-4f73-8214-e4ed0fa6404e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecf6642b7b84d3c9756bb41ab3349d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:42.421410 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0dabda-f9fc-4f73-8214-e4ed0fa6404e service cinder] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:42.422575 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0dabda-f9fc-4f73-8214-e4ed0fa6404e service cinder] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:42.423020 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0dabda-f9fc-4f73-8214-e4ed0fa6404e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:42.434384 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0dabda-f9fc-4f73-8214-e4ed0fa6404e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:42.434384 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0dabda-f9fc-4f73-8214-e4ed0fa6404e service cinder] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:42.445625 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1144/2297] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:40:42 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:40:47.253868 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45744e62-7928-4a1e-a0ee-8daec02a1439 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:47.317997 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45744e62-7928-4a1e-a0ee-8daec02a1439 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:47.322493 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5224a511-38c9-40e7-b4d4-d54da18d4e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:47.323124 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5224a511-38c9-40e7-b4d4-d54da18d4e6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:47.323407 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5224a511-38c9-40e7-b4d4-d54da18d4e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:47.323684 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5224a511-38c9-40e7-b4d4-d54da18d4e6d admin admin] PATH_INFO: `/v3/users/ffc376ff160d45e4964c34d59ade6c5b` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:47.324811 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5224a511-38c9-40e7-b4d4-d54da18d4e6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffc376ff160d45e4964c34d59ade6c5b)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:47.326783 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5224a511-38c9-40e7-b4d4-d54da18d4e6d admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:47.590803 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-5224a511-38c9-40e7-b4d4-d54da18d4e6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffc376ff160d45e4964c34d59ade6c5b'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:47.665324 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-5224a511-38c9-40e7-b4d4-d54da18d4e6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffc376ff160d45e4964c34d59ade6c5b'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:47.666050 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-5224a511-38c9-40e7-b4d4-d54da18d4e6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffc376ff160d45e4964c34d59ade6c5b'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:47.666798 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-5224a511-38c9-40e7-b4d4-d54da18d4e6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffc376ff160d45e4964c34d59ade6c5b'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:47.679669 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-5224a511-38c9-40e7-b4d4-d54da18d4e6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffc376ff160d45e4964c34d59ade6c5b'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:47.691139 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:INSERT=2,keystone:SELECT=89,keystone:DELETE=9 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:40:47.702791 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1154/2298] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:40:47 2022] DELETE /identity/v3/users/ffc376ff160d45e4964c34d59ade6c5b => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:40:47.711355 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8ce13c-320d-4e93-9810-42b2e32d423f None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:47.711873 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8ce13c-320d-4e93-9810-42b2e32d423f None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:47.711976 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8ce13c-320d-4e93-9810-42b2e32d423f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:47.733765 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-cf8ce13c-320d-4e93-9810-42b2e32d423f None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:40:47.734971 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-cf8ce13c-320d-4e93-9810-42b2e32d423f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:47.795093 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1145/2299] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:40:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:40:47.804816 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5224a511-38c9-40e7-b4d4-d54da18d4e6d admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:47.806291 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-5224a511-38c9-40e7-b4d4-d54da18d4e6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:47.864703 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5224a511-38c9-40e7-b4d4-d54da18d4e6d admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:47.868140 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c8273f-95fd-4167-b518-3dc47059108d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:47.868740 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c8273f-95fd-4167-b518-3dc47059108d service neutron] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:47.869014 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c8273f-95fd-4167-b518-3dc47059108d service neutron] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:47.869286 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c8273f-95fd-4167-b518-3dc47059108d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:47.878473 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c8273f-95fd-4167-b518-3dc47059108d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:47.879894 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c8273f-95fd-4167-b518-3dc47059108d service neutron] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:47.889097 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1155/2300] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:40:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:40:48.336299 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8ce13c-320d-4e93-9810-42b2e32d423f None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:48.338095 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-cf8ce13c-320d-4e93-9810-42b2e32d423f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:48.354016 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8ce13c-320d-4e93-9810-42b2e32d423f None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:48.362530 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd6db16-e210-4dcc-8248-72d919ea13cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:48.363195 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd6db16-e210-4dcc-8248-72d919ea13cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:48.363476 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd6db16-e210-4dcc-8248-72d919ea13cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:48.363752 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd6db16-e210-4dcc-8248-72d919ea13cf admin admin] PATH_INFO: `/v3/projects/4ae2b75b233d499b8f9213b6a061c159` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:48.364898 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd6db16-e210-4dcc-8248-72d919ea13cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ae2b75b233d499b8f9213b6a061c159)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:48.367067 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd6db16-e210-4dcc-8248-72d919ea13cf admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:48.476598 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-afd6db16-e210-4dcc-8248-72d919ea13cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ae2b75b233d499b8f9213b6a061c159', 'request_id': 'req-afd6db16-e210-4dcc-8248-72d919ea13cf'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:48.483540 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-afd6db16-e210-4dcc-8248-72d919ea13cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ae2b75b233d499b8f9213b6a061c159', 'request_id': 'req-afd6db16-e210-4dcc-8248-72d919ea13cf'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:48.550961 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-afd6db16-e210-4dcc-8248-72d919ea13cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ae2b75b233d499b8f9213b6a061c159', 'request_id': 'req-afd6db16-e210-4dcc-8248-72d919ea13cf'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:48.563160 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-afd6db16-e210-4dcc-8248-72d919ea13cf admin admin] The token cache is being invalidate because project 4ae2b75b233d499b8f9213b6a061c159 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:40:48.563456 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-afd6db16-e210-4dcc-8248-72d919ea13cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:48.564110 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-afd6db16-e210-4dcc-8248-72d919ea13cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:40:48.565725 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1146/2301] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:40:48 2022] DELETE /identity/v3/projects/4ae2b75b233d499b8f9213b6a061c159 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:40:48.571866 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1660fd72-de49-4ac3-94b7-d554db8e1f8c None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:48.572429 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1660fd72-de49-4ac3-94b7-d554db8e1f8c None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:48.572618 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1660fd72-de49-4ac3-94b7-d554db8e1f8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:48.594758 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-1660fd72-de49-4ac3-94b7-d554db8e1f8c None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:40:48.596038 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-1660fd72-de49-4ac3-94b7-d554db8e1f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:48.678669 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1156/2302] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:40:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:40:48.688613 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962c35b2-673e-4051-8035-232e631330a5 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:48.688908 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962c35b2-673e-4051-8035-232e631330a5 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:48.689192 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962c35b2-673e-4051-8035-232e631330a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:48.728588 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-962c35b2-673e-4051-8035-232e631330a5 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:40:48.729856 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-962c35b2-673e-4051-8035-232e631330a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:48.739513 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1147/2303] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:40:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:40:48.743927 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1660fd72-de49-4ac3-94b7-d554db8e1f8c None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:48.754912 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1660fd72-de49-4ac3-94b7-d554db8e1f8c None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:48.759475 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75e1992-ab5d-4f84-b579-658411ade90e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:48.760077 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75e1992-ab5d-4f84-b579-658411ade90e admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:48.760356 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75e1992-ab5d-4f84-b579-658411ade90e admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:48.760623 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75e1992-ab5d-4f84-b579-658411ade90e admin admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:48.761709 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75e1992-ab5d-4f84-b579-658411ade90e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:40:48.762016 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75e1992-ab5d-4f84-b579-658411ade90e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:48.763612 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75e1992-ab5d-4f84-b579-658411ade90e admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:48.775665 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1157/2304] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 15 07:40:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:40:48.780061 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962c35b2-673e-4051-8035-232e631330a5 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:48.790212 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962c35b2-673e-4051-8035-232e631330a5 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:48.795118 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0e82af-bceb-43e3-bdae-88dad55d6978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:48.795735 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0e82af-bceb-43e3-bdae-88dad55d6978 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:48.796529 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0e82af-bceb-43e3-bdae-88dad55d6978 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:48.796806 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0e82af-bceb-43e3-bdae-88dad55d6978 admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:48.797837 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0e82af-bceb-43e3-bdae-88dad55d6978 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:48.800207 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0e82af-bceb-43e3-bdae-88dad55d6978 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:48.889779 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1148/2305] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:40:48 2022] POST /identity/v3/projects => generated 381 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:40:48.893744 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75e1992-ab5d-4f84-b579-658411ade90e admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:48.907659 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75e1992-ab5d-4f84-b579-658411ade90e admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:48.977496 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176093b7-256a-4ad9-b37a-4b65b9747147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:48.978317 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176093b7-256a-4ad9-b37a-4b65b9747147 admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:48.978682 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176093b7-256a-4ad9-b37a-4b65b9747147 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:48.978967 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176093b7-256a-4ad9-b37a-4b65b9747147 admin admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:48.979617 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176093b7-256a-4ad9-b37a-4b65b9747147 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:48.981539 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176093b7-256a-4ad9-b37a-4b65b9747147 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:49.085596 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1158/2306] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:40:48 2022] POST /identity/v3/users => generated 291 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:40:49.089637 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0e82af-bceb-43e3-bdae-88dad55d6978 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:49.100982 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0e82af-bceb-43e3-bdae-88dad55d6978 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:49.105733 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88cb377-e53a-463d-929d-83a9728c6508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:49.106322 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88cb377-e53a-463d-929d-83a9728c6508 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:49.106643 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88cb377-e53a-463d-929d-83a9728c6508 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:49.106877 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88cb377-e53a-463d-929d-83a9728c6508 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:49.107519 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88cb377-e53a-463d-929d-83a9728c6508 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:49.109513 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88cb377-e53a-463d-929d-83a9728c6508 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:49.119509 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1149/2307] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:40:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:40:49.123526 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176093b7-256a-4ad9-b37a-4b65b9747147 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:49.135746 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176093b7-256a-4ad9-b37a-4b65b9747147 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:49.142576 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84a50c8-4ac1-4bac-a7a5-fc791f3c9042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:49.142576 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84a50c8-4ac1-4bac-a7a5-fc791f3c9042 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:49.143372 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84a50c8-4ac1-4bac-a7a5-fc791f3c9042 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:49.143470 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84a50c8-4ac1-4bac-a7a5-fc791f3c9042 admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:49.144182 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84a50c8-4ac1-4bac-a7a5-fc791f3c9042 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:49.145887 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84a50c8-4ac1-4bac-a7a5-fc791f3c9042 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:49.152837 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:40:49.162857 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1159/2308] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:40:49 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 07:40:49.164308 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88cb377-e53a-463d-929d-83a9728c6508 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:49.181885 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88cb377-e53a-463d-929d-83a9728c6508 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:49.187667 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae797c2d-21dc-4f86-becb-bf05686f6c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:49.192985 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae797c2d-21dc-4f86-becb-bf05686f6c1c admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:49.192985 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae797c2d-21dc-4f86-becb-bf05686f6c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:49.193840 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae797c2d-21dc-4f86-becb-bf05686f6c1c admin admin] PATH_INFO: `/v3/projects/66cb9c7c7eb24e30bf89b3680c713472/users/ae7e3a9a28134a28a3f8f30d4b0fcb8c/roles/053e12af84e943e699460a9d76c2072b` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:49.212330 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae797c2d-21dc-4f86-becb-bf05686f6c1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=66cb9c7c7eb24e30bf89b3680c713472, user_id=ae7e3a9a28134a28a3f8f30d4b0fcb8c, role_id=053e12af84e943e699460a9d76c2072b)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:49.215925 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae797c2d-21dc-4f86-becb-bf05686f6c1c admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:49.220791 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:40:49.312412 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1150/2309] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:40:49 2022] PUT /identity/v3/projects/66cb9c7c7eb24e30bf89b3680c713472/users/ae7e3a9a28134a28a3f8f30d4b0fcb8c/roles/053e12af84e943e699460a9d76c2072b => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:40:49.317461 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3f7c7b-613e-4858-92f3-e22b79df60db None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:49.317799 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3f7c7b-613e-4858-92f3-e22b79df60db None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:49.318078 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3f7c7b-613e-4858-92f3-e22b79df60db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:49.345316 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-de3f7c7b-613e-4858-92f3-e22b79df60db None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:40:49.351048 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-de3f7c7b-613e-4858-92f3-e22b79df60db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:49.419525 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1160/2310] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:40:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:40:49.433965 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae797c2d-21dc-4f86-becb-bf05686f6c1c admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:49.438130 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-ae797c2d-21dc-4f86-becb-bf05686f6c1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:49.507533 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae797c2d-21dc-4f86-becb-bf05686f6c1c admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:49.511524 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9334af4d-5d59-4896-8397-181309a9e385 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:49.512139 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9334af4d-5d59-4896-8397-181309a9e385 service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:49.512416 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9334af4d-5d59-4896-8397-181309a9e385 service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:49.512687 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9334af4d-5d59-4896-8397-181309a9e385 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:49.529471 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9334af4d-5d59-4896-8397-181309a9e385 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:49.532434 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9334af4d-5d59-4896-8397-181309a9e385 service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:49.543874 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1151/2311] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:40:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 07:40:49.609761 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ac954-5790-4498-bb4d-fb32a5602f68 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:49.610353 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ac954-5790-4498-bb4d-fb32a5602f68 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:49.610588 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ac954-5790-4498-bb4d-fb32a5602f68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:49.632314 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-714ac954-5790-4498-bb4d-fb32a5602f68 None None] MFA Rules not processed for user `f9c68c4bd9074d8d9b893755e28c0807`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:40:49.633622 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-714ac954-5790-4498-bb4d-fb32a5602f68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:49.722508 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1161/2312] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Sep 15 07:40:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:40:49.767439 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9334af4d-5d59-4896-8397-181309a9e385 service neutron] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:49.768957 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-9334af4d-5d59-4896-8397-181309a9e385 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:49.850264 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9334af4d-5d59-4896-8397-181309a9e385 service neutron] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:49.855346 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589bc1a3-2dbf-4155-8110-d0908e717b80 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72b90d35087e4d7a80e9417082a1541f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:49.855954 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589bc1a3-2dbf-4155-8110-d0908e717b80 service swift] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:49.856234 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589bc1a3-2dbf-4155-8110-d0908e717b80 service swift] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:49.856514 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589bc1a3-2dbf-4155-8110-d0908e717b80 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:49.866373 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589bc1a3-2dbf-4155-8110-d0908e717b80 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:49.868137 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589bc1a3-2dbf-4155-8110-d0908e717b80 service swift] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:49.877228 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1152/2313] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Sep 15 07:40:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 07:40:51.325149 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac33aa8b-1648-48da-b4d6-c80f34200210 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:51.326261 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac33aa8b-1648-48da-b4d6-c80f34200210 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:51.326974 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac33aa8b-1648-48da-b4d6-c80f34200210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:51.370502 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-ac33aa8b-1648-48da-b4d6-c80f34200210 None None] MFA Rules not processed for user `f9c68c4bd9074d8d9b893755e28c0807`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:40:51.374610 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-ac33aa8b-1648-48da-b4d6-c80f34200210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:51.407059 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4134d33c-33b2-461d-815e-7d8b0595e13d None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:51.409242 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4134d33c-33b2-461d-815e-7d8b0595e13d None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:51.409661 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4134d33c-33b2-461d-815e-7d8b0595e13d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:51.429296 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1162/2314] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Sep 15 07:40:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:40:51.431341 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac33aa8b-1648-48da-b4d6-c80f34200210 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:51.462174 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac33aa8b-1648-48da-b4d6-c80f34200210 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:51.477690 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744b123a-d401-4c54-ac93-ef9b936a7797 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72b90d35087e4d7a80e9417082a1541f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:51.478780 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744b123a-d401-4c54-ac93-ef9b936a7797 service swift] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:51.479106 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744b123a-d401-4c54-ac93-ef9b936a7797 service swift] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:51.479865 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744b123a-d401-4c54-ac93-ef9b936a7797 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:51.486632 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-4134d33c-33b2-461d-815e-7d8b0595e13d None None] MFA Rules not processed for user `ae7e3a9a28134a28a3f8f30d4b0fcb8c`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:40:51.492765 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-4134d33c-33b2-461d-815e-7d8b0595e13d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:51.505797 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744b123a-d401-4c54-ac93-ef9b936a7797 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:51.509660 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744b123a-d401-4c54-ac93-ef9b936a7797 service swift] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:51.533942 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1163/2315] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Sep 15 07:40:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 07:40:51.614114 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1153/2316] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:40:51 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:40:51.626649 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744b123a-d401-4c54-ac93-ef9b936a7797 service swift] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:51.634029 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-744b123a-d401-4c54-ac93-ef9b936a7797 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:51.762623 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744b123a-d401-4c54-ac93-ef9b936a7797 service swift] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:51.767204 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691d4c68-9e80-4e27-96f1-fabebbcfb4cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecf6642b7b84d3c9756bb41ab3349d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:51.767825 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691d4c68-9e80-4e27-96f1-fabebbcfb4cb service cinder] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:51.768112 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691d4c68-9e80-4e27-96f1-fabebbcfb4cb service cinder] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:51.768384 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691d4c68-9e80-4e27-96f1-fabebbcfb4cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:51.781778 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691d4c68-9e80-4e27-96f1-fabebbcfb4cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:51.783351 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691d4c68-9e80-4e27-96f1-fabebbcfb4cb service cinder] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:51.797774 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1164/2317] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:40:51 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 07:40:52.999216 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ab5af3-3789-4c3c-ba7b-90ba9a286f14 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:53.001746 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ab5af3-3789-4c3c-ba7b-90ba9a286f14 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:53.001746 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ab5af3-3789-4c3c-ba7b-90ba9a286f14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:53.024093 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-22ab5af3-3789-4c3c-ba7b-90ba9a286f14 None None] MFA Rules not processed for user `f9c68c4bd9074d8d9b893755e28c0807`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:40:53.025814 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-22ab5af3-3789-4c3c-ba7b-90ba9a286f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:53.039181 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1154/2318] 10.222.0.42 () {58 vars in 967 bytes} [Thu Sep 15 07:40:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:40:53.064222 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691d4c68-9e80-4e27-96f1-fabebbcfb4cb service cinder] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:53.077602 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691d4c68-9e80-4e27-96f1-fabebbcfb4cb service cinder] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:53.089427 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8040139-e333-4243-82ce-d868ddb44af1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72b90d35087e4d7a80e9417082a1541f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:53.090332 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8040139-e333-4243-82ce-d868ddb44af1 service swift] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:53.090774 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8040139-e333-4243-82ce-d868ddb44af1 service swift] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:53.091241 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8040139-e333-4243-82ce-d868ddb44af1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:53.103023 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8040139-e333-4243-82ce-d868ddb44af1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:53.104834 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8040139-e333-4243-82ce-d868ddb44af1 service swift] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:53.115528 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1165/2319] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Sep 15 07:40:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 07:40:53.714481 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a34ea9-2484-4d99-b2e0-84fe06372970 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:53.715078 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a34ea9-2484-4d99-b2e0-84fe06372970 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:53.715366 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a34ea9-2484-4d99-b2e0-84fe06372970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:53.735714 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-67a34ea9-2484-4d99-b2e0-84fe06372970 None None] MFA Rules not processed for user `ae7e3a9a28134a28a3f8f30d4b0fcb8c`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:40:53.736925 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-67a34ea9-2484-4d99-b2e0-84fe06372970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:53.743740 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1155/2320] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:40:53 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:40:54.404936 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353c146f-ae5f-4785-9858-9b47ca605364 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:54.406670 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353c146f-ae5f-4785-9858-9b47ca605364 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:54.407469 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353c146f-ae5f-4785-9858-9b47ca605364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:54.435019 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-353c146f-ae5f-4785-9858-9b47ca605364 None None] MFA Rules not processed for user `f9c68c4bd9074d8d9b893755e28c0807`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:40:54.436527 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-353c146f-ae5f-4785-9858-9b47ca605364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:54.447191 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1166/2321] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Sep 15 07:40:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 07:40:54.463218 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a34ea9-2484-4d99-b2e0-84fe06372970 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:54.473109 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:40:54.486946 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a34ea9-2484-4d99-b2e0-84fe06372970 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:54.489250 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704dbd2e-0eea-47b1-995b-82e7d05c6777 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72b90d35087e4d7a80e9417082a1541f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:54.490209 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704dbd2e-0eea-47b1-995b-82e7d05c6777 service swift] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:54.490740 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704dbd2e-0eea-47b1-995b-82e7d05c6777 service swift] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:54.491726 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704dbd2e-0eea-47b1-995b-82e7d05c6777 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:54.504771 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704dbd2e-0eea-47b1-995b-82e7d05c6777 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:54.507197 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704dbd2e-0eea-47b1-995b-82e7d05c6777 service swift] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:54.517398 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1156/2322] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Sep 15 07:40:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 07:40:55.525426 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a95decb-613c-4ffa-9eb8-660c6ab9e23e None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:55.526033 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a95decb-613c-4ffa-9eb8-660c6ab9e23e None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:55.526137 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a95decb-613c-4ffa-9eb8-660c6ab9e23e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:55.561795 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-4a95decb-613c-4ffa-9eb8-660c6ab9e23e None None] MFA Rules not processed for user `f9c68c4bd9074d8d9b893755e28c0807`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:40:55.563673 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-4a95decb-613c-4ffa-9eb8-660c6ab9e23e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:55.582914 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1167/2323] 10.222.0.42 () {58 vars in 967 bytes} [Thu Sep 15 07:40:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:40:55.600836 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704dbd2e-0eea-47b1-995b-82e7d05c6777 service swift] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:55.613629 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704dbd2e-0eea-47b1-995b-82e7d05c6777 service swift] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:55.619906 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ae266d-aa45-48cc-88dd-fcfeacc71968 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72b90d35087e4d7a80e9417082a1541f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:55.620441 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ae266d-aa45-48cc-88dd-fcfeacc71968 service swift] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:55.621052 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ae266d-aa45-48cc-88dd-fcfeacc71968 service swift] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:55.621519 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ae266d-aa45-48cc-88dd-fcfeacc71968 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:55.635798 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ae266d-aa45-48cc-88dd-fcfeacc71968 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:55.637436 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ae266d-aa45-48cc-88dd-fcfeacc71968 service swift] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:55.650341 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1157/2324] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Sep 15 07:40:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 07:40:59.730930 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a95decb-613c-4ffa-9eb8-660c6ab9e23e None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:40:59.732388 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-4a95decb-613c-4ffa-9eb8-660c6ab9e23e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:40:59.748253 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:40:59.764248 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a95decb-613c-4ffa-9eb8-660c6ab9e23e None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:40:59.772129 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc91c0d-4447-4b05-ac45-fb32d1d9df52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:40:59.773151 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc91c0d-4447-4b05-ac45-fb32d1d9df52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:40:59.773480 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc91c0d-4447-4b05-ac45-fb32d1d9df52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:40:59.773928 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc91c0d-4447-4b05-ac45-fb32d1d9df52 admin admin] PATH_INFO: `/v3/users/82e55686994a4e6b86096cc21c52085d` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:40:59.776429 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc91c0d-4447-4b05-ac45-fb32d1d9df52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82e55686994a4e6b86096cc21c52085d)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:40:59.779872 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc91c0d-4447-4b05-ac45-fb32d1d9df52 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:40:59.990765 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-3fc91c0d-4447-4b05-ac45-fb32d1d9df52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82e55686994a4e6b86096cc21c52085d'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:41:00.098277 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-3fc91c0d-4447-4b05-ac45-fb32d1d9df52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82e55686994a4e6b86096cc21c52085d'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:41:00.099122 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-3fc91c0d-4447-4b05-ac45-fb32d1d9df52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82e55686994a4e6b86096cc21c52085d'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:41:00.099816 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-3fc91c0d-4447-4b05-ac45-fb32d1d9df52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82e55686994a4e6b86096cc21c52085d'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:41:00.109263 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-3fc91c0d-4447-4b05-ac45-fb32d1d9df52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82e55686994a4e6b86096cc21c52085d'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:41:00.160901 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1168/2325] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:40:59 2022] DELETE /identity/v3/users/82e55686994a4e6b86096cc21c52085d => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:41:00.162345 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7067a44-6725-4955-8669-b3a8e3654cd7 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:41:00.165458 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7067a44-6725-4955-8669-b3a8e3654cd7 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:41:00.165771 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7067a44-6725-4955-8669-b3a8e3654cd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:41:00.208965 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-d7067a44-6725-4955-8669-b3a8e3654cd7 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:41:00.210104 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-d7067a44-6725-4955-8669-b3a8e3654cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:41:00.293741 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1158/2326] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:41:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:41:00.305222 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc91c0d-4447-4b05-ac45-fb32d1d9df52 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:41:00.306874 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-3fc91c0d-4447-4b05-ac45-fb32d1d9df52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:41:00.394945 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc91c0d-4447-4b05-ac45-fb32d1d9df52 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:41:00.401632 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d395ff3d-3a8c-4825-ac8b-889e36bfd868 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:41:00.402775 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d395ff3d-3a8c-4825-ac8b-889e36bfd868 service neutron] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:41:00.403097 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d395ff3d-3a8c-4825-ac8b-889e36bfd868 service neutron] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:41:00.403097 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d395ff3d-3a8c-4825-ac8b-889e36bfd868 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:41:00.415421 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d395ff3d-3a8c-4825-ac8b-889e36bfd868 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:41:00.419029 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d395ff3d-3a8c-4825-ac8b-889e36bfd868 service neutron] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:41:00.429916 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1169/2327] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:41:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:41:00.785129 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7067a44-6725-4955-8669-b3a8e3654cd7 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:41:00.790186 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-d7067a44-6725-4955-8669-b3a8e3654cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:41:00.811447 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7067a44-6725-4955-8669-b3a8e3654cd7 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:41:00.817719 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ba2c21-8fe6-41ad-b7b8-f5730fe05450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:41:00.818375 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ba2c21-8fe6-41ad-b7b8-f5730fe05450 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:41:00.818662 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ba2c21-8fe6-41ad-b7b8-f5730fe05450 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:41:00.818941 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ba2c21-8fe6-41ad-b7b8-f5730fe05450 admin admin] PATH_INFO: `/v3/projects/e0730e401f844b55a78df53dba8e4b5c` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:41:00.820503 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ba2c21-8fe6-41ad-b7b8-f5730fe05450 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0730e401f844b55a78df53dba8e4b5c)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:41:00.822945 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ba2c21-8fe6-41ad-b7b8-f5730fe05450 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:41:00.945533 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-32ba2c21-8fe6-41ad-b7b8-f5730fe05450 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0730e401f844b55a78df53dba8e4b5c', 'request_id': 'req-32ba2c21-8fe6-41ad-b7b8-f5730fe05450'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:41:00.956134 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-32ba2c21-8fe6-41ad-b7b8-f5730fe05450 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0730e401f844b55a78df53dba8e4b5c', 'request_id': 'req-32ba2c21-8fe6-41ad-b7b8-f5730fe05450'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:41:01.032547 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-32ba2c21-8fe6-41ad-b7b8-f5730fe05450 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0730e401f844b55a78df53dba8e4b5c', 'request_id': 'req-32ba2c21-8fe6-41ad-b7b8-f5730fe05450'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:41:01.049211 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-32ba2c21-8fe6-41ad-b7b8-f5730fe05450 admin admin] The token cache is being invalidate because project e0730e401f844b55a78df53dba8e4b5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:41:01.049692 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-32ba2c21-8fe6-41ad-b7b8-f5730fe05450 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:41:01.050456 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-32ba2c21-8fe6-41ad-b7b8-f5730fe05450 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:41:01.052635 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1159/2328] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:41:00 2022] DELETE /identity/v3/projects/e0730e401f844b55a78df53dba8e4b5c => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:41:01.057880 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b595cc8-bb5f-4978-9003-b8a028c9d0f5 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:41:01.058371 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b595cc8-bb5f-4978-9003-b8a028c9d0f5 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:41:01.058725 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b595cc8-bb5f-4978-9003-b8a028c9d0f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:41:01.087371 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-7b595cc8-bb5f-4978-9003-b8a028c9d0f5 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:41:01.088799 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-7b595cc8-bb5f-4978-9003-b8a028c9d0f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:41:01.175981 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1170/2329] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:41:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:41:01.177822 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaae152-bc03-4da0-bb61-26b731769566 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:41:01.178300 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaae152-bc03-4da0-bb61-26b731769566 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:41:01.178613 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaae152-bc03-4da0-bb61-26b731769566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:41:01.208293 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-6aaae152-bc03-4da0-bb61-26b731769566 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:41:01.210794 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-6aaae152-bc03-4da0-bb61-26b731769566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:41:01.219988 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1160/2330] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:41:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:41:01.224753 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b595cc8-bb5f-4978-9003-b8a028c9d0f5 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:41:01.235581 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b595cc8-bb5f-4978-9003-b8a028c9d0f5 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:41:01.240512 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019d6aa0-3010-4a26-8278-e147deb31060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:41:01.241127 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019d6aa0-3010-4a26-8278-e147deb31060 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:41:01.241410 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019d6aa0-3010-4a26-8278-e147deb31060 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:41:01.241732 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019d6aa0-3010-4a26-8278-e147deb31060 admin admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:41:01.242394 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019d6aa0-3010-4a26-8278-e147deb31060 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:41:01.242837 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019d6aa0-3010-4a26-8278-e147deb31060 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:41:01.244399 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019d6aa0-3010-4a26-8278-e147deb31060 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:41:01.258342 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1171/2331] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 15 07:41:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:41:01.262348 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaae152-bc03-4da0-bb61-26b731769566 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:41:01.272539 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaae152-bc03-4da0-bb61-26b731769566 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:41:01.279981 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e04157-7452-4ef0-b14c-fe2b41226173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:41:01.280856 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e04157-7452-4ef0-b14c-fe2b41226173 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:41:01.282186 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e04157-7452-4ef0-b14c-fe2b41226173 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:41:01.282789 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e04157-7452-4ef0-b14c-fe2b41226173 admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:41:01.283612 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e04157-7452-4ef0-b14c-fe2b41226173 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:41:01.285741 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e04157-7452-4ef0-b14c-fe2b41226173 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:41:01.390684 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1161/2332] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:41:01 2022] POST /identity/v3/projects => generated 377 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:41:01.391885 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019d6aa0-3010-4a26-8278-e147deb31060 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:41:01.401097 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:41:01.409694 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019d6aa0-3010-4a26-8278-e147deb31060 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:41:01.507834 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559997b3-cc09-4d83-85ca-7e91cb1f612c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:41:01.508430 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559997b3-cc09-4d83-85ca-7e91cb1f612c admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:41:01.508730 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559997b3-cc09-4d83-85ca-7e91cb1f612c admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:41:01.508978 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559997b3-cc09-4d83-85ca-7e91cb1f612c admin admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:41:01.509640 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559997b3-cc09-4d83-85ca-7e91cb1f612c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:41:01.511727 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559997b3-cc09-4d83-85ca-7e91cb1f612c admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:41:01.652974 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1172/2333] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:41:01 2022] POST /identity/v3/users => generated 289 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:41:01.657820 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e04157-7452-4ef0-b14c-fe2b41226173 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:41:01.673364 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e04157-7452-4ef0-b14c-fe2b41226173 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:41:01.675712 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528c8a2e-cc7c-481e-8b0a-baa28dfc1914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:41:01.676343 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528c8a2e-cc7c-481e-8b0a-baa28dfc1914 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:41:01.676652 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528c8a2e-cc7c-481e-8b0a-baa28dfc1914 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:41:01.676928 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528c8a2e-cc7c-481e-8b0a-baa28dfc1914 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:41:01.677603 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528c8a2e-cc7c-481e-8b0a-baa28dfc1914 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:41:01.679277 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528c8a2e-cc7c-481e-8b0a-baa28dfc1914 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:41:01.695657 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1162/2334] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:41:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:41:01.696840 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559997b3-cc09-4d83-85ca-7e91cb1f612c admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:41:01.709225 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559997b3-cc09-4d83-85ca-7e91cb1f612c admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:41:01.714464 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee64fbef-fdc1-49ba-ae3e-5c53a7df468c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:41:01.715263 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee64fbef-fdc1-49ba-ae3e-5c53a7df468c admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:41:01.715625 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee64fbef-fdc1-49ba-ae3e-5c53a7df468c admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:41:01.715963 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee64fbef-fdc1-49ba-ae3e-5c53a7df468c admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:41:01.716682 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee64fbef-fdc1-49ba-ae3e-5c53a7df468c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:41:01.718548 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee64fbef-fdc1-49ba-ae3e-5c53a7df468c admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:41:01.730925 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1173/2335] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:41:01 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:41:01.735484 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528c8a2e-cc7c-481e-8b0a-baa28dfc1914 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:41:01.751531 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528c8a2e-cc7c-481e-8b0a-baa28dfc1914 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:41:01.757244 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efe6405-a907-4db0-8b9c-0277209408f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:41:01.757945 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efe6405-a907-4db0-8b9c-0277209408f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:41:01.758230 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efe6405-a907-4db0-8b9c-0277209408f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:41:01.758504 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efe6405-a907-4db0-8b9c-0277209408f9 admin admin] PATH_INFO: `/v3/projects/af1e008c4bfa41e18488c63ea0f17b27/users/ebd85e42855448268c89a200f4392316/roles/053e12af84e943e699460a9d76c2072b` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:41:01.785295 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efe6405-a907-4db0-8b9c-0277209408f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af1e008c4bfa41e18488c63ea0f17b27, user_id=ebd85e42855448268c89a200f4392316, role_id=053e12af84e943e699460a9d76c2072b)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:41:01.788949 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efe6405-a907-4db0-8b9c-0277209408f9 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:41:01.881372 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:41:01.890458 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1163/2336] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:41:01 2022] PUT /identity/v3/projects/af1e008c4bfa41e18488c63ea0f17b27/users/ebd85e42855448268c89a200f4392316/roles/053e12af84e943e699460a9d76c2072b => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 07:41:01.895928 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3914a4-7894-4f42-a22c-9aaec429abe2 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:41:01.896213 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3914a4-7894-4f42-a22c-9aaec429abe2 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:41:01.896495 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3914a4-7894-4f42-a22c-9aaec429abe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:41:01.920969 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-2b3914a4-7894-4f42-a22c-9aaec429abe2 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:41:01.925646 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-2b3914a4-7894-4f42-a22c-9aaec429abe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:41:02.024181 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1174/2337] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:41:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:41:02.035272 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efe6405-a907-4db0-8b9c-0277209408f9 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:41:02.046705 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-8efe6405-a907-4db0-8b9c-0277209408f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:41:02.133279 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efe6405-a907-4db0-8b9c-0277209408f9 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:41:02.137727 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43320952-649e-4751-9531-f8e92527f6d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:41:02.138348 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43320952-649e-4751-9531-f8e92527f6d1 service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:41:02.138637 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43320952-649e-4751-9531-f8e92527f6d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:41:02.138913 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43320952-649e-4751-9531-f8e92527f6d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:41:02.150596 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43320952-649e-4751-9531-f8e92527f6d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:41:02.152088 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43320952-649e-4751-9531-f8e92527f6d1 service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:41:02.181122 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1164/2338] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:41:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:41:08.544242 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3914a4-7894-4f42-a22c-9aaec429abe2 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:41:08.546076 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-2b3914a4-7894-4f42-a22c-9aaec429abe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:41:08.562109 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3914a4-7894-4f42-a22c-9aaec429abe2 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:41:08.570663 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafb9ed6-fadc-4848-bbb6-9c156cbb87d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:41:08.571724 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafb9ed6-fadc-4848-bbb6-9c156cbb87d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:41:08.572160 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafb9ed6-fadc-4848-bbb6-9c156cbb87d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:41:08.572625 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafb9ed6-fadc-4848-bbb6-9c156cbb87d4 admin admin] PATH_INFO: `/v3/users/ce0a1d3c36cb46a6947983b223e20b1b` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:41:08.574575 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafb9ed6-fadc-4848-bbb6-9c156cbb87d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce0a1d3c36cb46a6947983b223e20b1b)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:41:08.578238 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafb9ed6-fadc-4848-bbb6-9c156cbb87d4 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:41:08.818953 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-cafb9ed6-fadc-4848-bbb6-9c156cbb87d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce0a1d3c36cb46a6947983b223e20b1b'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:41:08.926545 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-cafb9ed6-fadc-4848-bbb6-9c156cbb87d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce0a1d3c36cb46a6947983b223e20b1b'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:41:08.927343 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-cafb9ed6-fadc-4848-bbb6-9c156cbb87d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce0a1d3c36cb46a6947983b223e20b1b'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:41:08.928049 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-cafb9ed6-fadc-4848-bbb6-9c156cbb87d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce0a1d3c36cb46a6947983b223e20b1b'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:41:08.958507 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-cafb9ed6-fadc-4848-bbb6-9c156cbb87d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce0a1d3c36cb46a6947983b223e20b1b'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:41:08.979308 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:41:09.002479 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1175/2339] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:41:08 2022] DELETE /identity/v3/users/ce0a1d3c36cb46a6947983b223e20b1b => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:41:09.005618 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9ded95-ff3a-4c15-a223-b9d245d90ced None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:41:09.006105 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9ded95-ff3a-4c15-a223-b9d245d90ced None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:41:09.006198 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9ded95-ff3a-4c15-a223-b9d245d90ced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:41:09.043641 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-2b9ded95-ff3a-4c15-a223-b9d245d90ced None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:41:09.045090 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-2b9ded95-ff3a-4c15-a223-b9d245d90ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:41:09.140342 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1165/2340] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:41:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:41:09.146719 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafb9ed6-fadc-4848-bbb6-9c156cbb87d4 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:41:09.148240 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-cafb9ed6-fadc-4848-bbb6-9c156cbb87d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:41:09.227690 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafb9ed6-fadc-4848-bbb6-9c156cbb87d4 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:41:09.232690 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdbe05d-2996-4a69-929d-861a4f4678f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:41:09.233470 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdbe05d-2996-4a69-929d-861a4f4678f6 service neutron] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:41:09.234061 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdbe05d-2996-4a69-929d-861a4f4678f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:41:09.234389 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdbe05d-2996-4a69-929d-861a4f4678f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:41:09.256167 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdbe05d-2996-4a69-929d-861a4f4678f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:41:09.257967 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdbe05d-2996-4a69-929d-861a4f4678f6 service neutron] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:41:09.280891 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1176/2341] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:41:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 07:41:10.176470 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9ded95-ff3a-4c15-a223-b9d245d90ced None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:41:10.177979 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-2b9ded95-ff3a-4c15-a223-b9d245d90ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:41:10.196974 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9ded95-ff3a-4c15-a223-b9d245d90ced None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:41:10.203165 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59efa3f1-59ce-42c4-8d9a-13f897b5b2a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:41:10.207750 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59efa3f1-59ce-42c4-8d9a-13f897b5b2a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:41:10.208245 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59efa3f1-59ce-42c4-8d9a-13f897b5b2a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:41:10.208675 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59efa3f1-59ce-42c4-8d9a-13f897b5b2a9 admin admin] PATH_INFO: `/v3/projects/a53d9cbe31d5495c9bb78d0e7fd5522e` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:41:10.210583 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59efa3f1-59ce-42c4-8d9a-13f897b5b2a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a53d9cbe31d5495c9bb78d0e7fd5522e)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:41:10.212906 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59efa3f1-59ce-42c4-8d9a-13f897b5b2a9 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:41:10.382264 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-59efa3f1-59ce-42c4-8d9a-13f897b5b2a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a53d9cbe31d5495c9bb78d0e7fd5522e', 'request_id': 'req-59efa3f1-59ce-42c4-8d9a-13f897b5b2a9'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:41:10.390669 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-59efa3f1-59ce-42c4-8d9a-13f897b5b2a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a53d9cbe31d5495c9bb78d0e7fd5522e', 'request_id': 'req-59efa3f1-59ce-42c4-8d9a-13f897b5b2a9'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:41:10.474847 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-59efa3f1-59ce-42c4-8d9a-13f897b5b2a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a53d9cbe31d5495c9bb78d0e7fd5522e', 'request_id': 'req-59efa3f1-59ce-42c4-8d9a-13f897b5b2a9'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:41:10.493190 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5de8e49-1d08-4b08-b128-1ff1d359d0b1 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:41:10.494575 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5de8e49-1d08-4b08-b128-1ff1d359d0b1 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:41:10.495338 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5de8e49-1d08-4b08-b128-1ff1d359d0b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:41:10.503087 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-59efa3f1-59ce-42c4-8d9a-13f897b5b2a9 admin admin] The token cache is being invalidate because project a53d9cbe31d5495c9bb78d0e7fd5522e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:41:10.503482 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-59efa3f1-59ce-42c4-8d9a-13f897b5b2a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:41:10.504128 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-59efa3f1-59ce-42c4-8d9a-13f897b5b2a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:41:10.515126 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1166/2342] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:41:10 2022] DELETE /identity/v3/projects/a53d9cbe31d5495c9bb78d0e7fd5522e => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:41:10.520594 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161bf22c-30ab-4d2a-bc18-2b8a0659f210 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:41:10.520987 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161bf22c-30ab-4d2a-bc18-2b8a0659f210 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:41:10.521337 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161bf22c-30ab-4d2a-bc18-2b8a0659f210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:41:10.557505 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-161bf22c-30ab-4d2a-bc18-2b8a0659f210 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:41:10.559327 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-161bf22c-30ab-4d2a-bc18-2b8a0659f210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:41:10.577078 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-c5de8e49-1d08-4b08-b128-1ff1d359d0b1 None None] MFA Rules not processed for user `ebd85e42855448268c89a200f4392316`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:41:10.579561 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-c5de8e49-1d08-4b08-b128-1ff1d359d0b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:41:10.584636 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:41:10.643348 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1167/2343] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:41:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:41:10.652615 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed706ddd-fb10-442b-b7ae-fbab66076cb6 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:41:10.652929 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed706ddd-fb10-442b-b7ae-fbab66076cb6 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:41:10.653174 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed706ddd-fb10-442b-b7ae-fbab66076cb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:41:10.668710 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1177/2344] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:41:10 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:41:10.673976 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5de8e49-1d08-4b08-b128-1ff1d359d0b1 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:41:10.675438 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-c5de8e49-1d08-4b08-b128-1ff1d359d0b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:41:10.680056 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-ed706ddd-fb10-442b-b7ae-fbab66076cb6 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:41:10.681314 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-ed706ddd-fb10-442b-b7ae-fbab66076cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:41:10.691522 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1168/2345] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:41:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:41:10.694740 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5de8e49-1d08-4b08-b128-1ff1d359d0b1 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:41:10.695974 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed706ddd-fb10-442b-b7ae-fbab66076cb6 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:41:10.700807 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb46a81a-3dc6-43ae-ba13-5738da051db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:41:10.701436 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb46a81a-3dc6-43ae-ba13-5738da051db0 admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:41:10.704137 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb46a81a-3dc6-43ae-ba13-5738da051db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:41:10.704137 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb46a81a-3dc6-43ae-ba13-5738da051db0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:41:10.704137 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb46a81a-3dc6-43ae-ba13-5738da051db0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:41:10.705409 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb46a81a-3dc6-43ae-ba13-5738da051db0 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:41:10.711203 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed706ddd-fb10-442b-b7ae-fbab66076cb6 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:41:10.716200 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7f074c-2644-46be-832c-138c6ac3e613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:41:10.716826 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7f074c-2644-46be-832c-138c6ac3e613 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:41:10.717109 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7f074c-2644-46be-832c-138c6ac3e613 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:41:10.717388 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7f074c-2644-46be-832c-138c6ac3e613 admin admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:41:10.718402 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7f074c-2644-46be-832c-138c6ac3e613 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:41:10.719933 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7f074c-2644-46be-832c-138c6ac3e613 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:41:10.721141 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7f074c-2644-46be-832c-138c6ac3e613 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:41:10.737244 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1169/2346] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 15 07:41:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:41:10.744606 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7f074c-2644-46be-832c-138c6ac3e613 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:41:10.752138 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7f074c-2644-46be-832c-138c6ac3e613 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:41:10.757025 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df412558-fc61-4953-bdbe-54966da2624f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:41:10.757681 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df412558-fc61-4953-bdbe-54966da2624f admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:41:10.757967 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df412558-fc61-4953-bdbe-54966da2624f admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:41:10.758239 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df412558-fc61-4953-bdbe-54966da2624f admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:41:10.758880 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df412558-fc61-4953-bdbe-54966da2624f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:41:10.760774 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df412558-fc61-4953-bdbe-54966da2624f admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:41:10.805084 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1178/2347] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:41:10 2022] POST /identity/v3/projects => generated 377 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:41:10.809409 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb46a81a-3dc6-43ae-ba13-5738da051db0 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:41:10.879413 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb46a81a-3dc6-43ae-ba13-5738da051db0 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:41:10.884259 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6725207d-76d6-4cee-a315-64d9d9fb18ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:41:10.884887 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6725207d-76d6-4cee-a315-64d9d9fb18ce admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:41:10.885170 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6725207d-76d6-4cee-a315-64d9d9fb18ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:41:10.885443 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6725207d-76d6-4cee-a315-64d9d9fb18ce admin admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:41:10.886111 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6725207d-76d6-4cee-a315-64d9d9fb18ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:41:10.888047 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6725207d-76d6-4cee-a315-64d9d9fb18ce admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:41:10.889775 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1170/2348] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:41:10 2022] POST /identity/v3/projects => generated 375 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 07:41:10.892565 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df412558-fc61-4953-bdbe-54966da2624f admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:41:10.899920 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:41:10.975409 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df412558-fc61-4953-bdbe-54966da2624f admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:41:10.981744 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07092086-19f4-4324-a23d-4f15e23588c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:41:10.982372 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07092086-19f4-4324-a23d-4f15e23588c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:41:10.982658 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07092086-19f4-4324-a23d-4f15e23588c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:41:10.982934 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07092086-19f4-4324-a23d-4f15e23588c7 admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:41:10.983567 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07092086-19f4-4324-a23d-4f15e23588c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:41:10.985839 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07092086-19f4-4324-a23d-4f15e23588c7 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:41:11.018623 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1179/2349] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:41:10 2022] POST /identity/v3/users => generated 295 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:41:11.029258 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6725207d-76d6-4cee-a315-64d9d9fb18ce admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:41:11.040467 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6725207d-76d6-4cee-a315-64d9d9fb18ce admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:41:11.047922 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee515ef-6497-4199-b148-101da5247198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:41:11.048542 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee515ef-6497-4199-b148-101da5247198 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:41:11.048823 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee515ef-6497-4199-b148-101da5247198 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:41:11.050737 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee515ef-6497-4199-b148-101da5247198 admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:41:11.050737 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee515ef-6497-4199-b148-101da5247198 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:41:11.051474 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee515ef-6497-4199-b148-101da5247198 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:41:11.065618 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1180/2350] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:41:11 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:41:11.069802 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee515ef-6497-4199-b148-101da5247198 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:41:11.084416 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee515ef-6497-4199-b148-101da5247198 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:41:11.089462 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476a0715-f999-4936-b7ad-fd377695f8ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:41:11.090458 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476a0715-f999-4936-b7ad-fd377695f8ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:41:11.090843 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476a0715-f999-4936-b7ad-fd377695f8ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:41:11.091264 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476a0715-f999-4936-b7ad-fd377695f8ba admin admin] PATH_INFO: `/v3/projects/e5ef5d2e9c5d4352ad39a7d1b1a0170e/users/1fb4689f1f2e48b2a52a1d05f1e98b30/roles/6bbb643e0d8d4bc0a9b27c26edf59216` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:41:11.108128 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1171/2351] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:41:10 2022] POST /identity/v3/users => generated 288 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 07:41:11.109379 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07092086-19f4-4324-a23d-4f15e23588c7 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:41:11.119051 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476a0715-f999-4936-b7ad-fd377695f8ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5ef5d2e9c5d4352ad39a7d1b1a0170e, user_id=1fb4689f1f2e48b2a52a1d05f1e98b30, role_id=6bbb643e0d8d4bc0a9b27c26edf59216)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:41:11.122310 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07092086-19f4-4324-a23d-4f15e23588c7 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:41:11.122473 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476a0715-f999-4936-b7ad-fd377695f8ba admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:41:11.127942 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db8276c-d26e-4c81-bc2a-0fefbadc97db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:41:11.128641 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db8276c-d26e-4c81-bc2a-0fefbadc97db admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:41:11.128849 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db8276c-d26e-4c81-bc2a-0fefbadc97db admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:41:11.129126 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db8276c-d26e-4c81-bc2a-0fefbadc97db admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:41:11.129803 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db8276c-d26e-4c81-bc2a-0fefbadc97db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:41:11.131479 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db8276c-d26e-4c81-bc2a-0fefbadc97db admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:41:11.149428 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1172/2352] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:41:11 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:41:11.150719 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db8276c-d26e-4c81-bc2a-0fefbadc97db admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:41:11.166871 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db8276c-d26e-4c81-bc2a-0fefbadc97db admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:41:11.172627 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92323f3-19d0-4b99-a96d-c0f61ba40b10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:41:11.173675 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92323f3-19d0-4b99-a96d-c0f61ba40b10 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:41:11.174151 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92323f3-19d0-4b99-a96d-c0f61ba40b10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:41:11.174505 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92323f3-19d0-4b99-a96d-c0f61ba40b10 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:41:11.175232 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92323f3-19d0-4b99-a96d-c0f61ba40b10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:41:11.178176 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92323f3-19d0-4b99-a96d-c0f61ba40b10 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:41:11.194174 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1173/2353] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:41:11 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:41:11.195447 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92323f3-19d0-4b99-a96d-c0f61ba40b10 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:41:11.206633 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92323f3-19d0-4b99-a96d-c0f61ba40b10 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:41:11.211711 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376dd4a0-c181-422d-bda6-0e0774a93083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:41:11.212432 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376dd4a0-c181-422d-bda6-0e0774a93083 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:41:11.212713 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376dd4a0-c181-422d-bda6-0e0774a93083 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:41:11.213000 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376dd4a0-c181-422d-bda6-0e0774a93083 admin admin] PATH_INFO: `/v3/projects/996835437ec740bfaa4cc5aa8e595b63/users/fe892a32557741aba2cfed60116c959e/roles/053e12af84e943e699460a9d76c2072b` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:41:11.230214 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376dd4a0-c181-422d-bda6-0e0774a93083 admin admin] RBAC: Authorizing `identity:create_grant(project_id=996835437ec740bfaa4cc5aa8e595b63, user_id=fe892a32557741aba2cfed60116c959e, role_id=053e12af84e943e699460a9d76c2072b)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:41:11.231922 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1181/2354] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:41:11 2022] PUT /identity/v3/projects/e5ef5d2e9c5d4352ad39a7d1b1a0170e/users/1fb4689f1f2e48b2a52a1d05f1e98b30/roles/6bbb643e0d8d4bc0a9b27c26edf59216 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:41:11.235632 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376dd4a0-c181-422d-bda6-0e0774a93083 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:41:11.236583 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476a0715-f999-4936-b7ad-fd377695f8ba admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:41:11.328219 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476a0715-f999-4936-b7ad-fd377695f8ba admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:41:11.338228 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1174/2355] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:41:11 2022] PUT /identity/v3/projects/996835437ec740bfaa4cc5aa8e595b63/users/fe892a32557741aba2cfed60116c959e/roles/053e12af84e943e699460a9d76c2072b => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 07:41:11.348481 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69ceee3-67ef-401b-be17-8be64cb49845 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:41:11.348776 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69ceee3-67ef-401b-be17-8be64cb49845 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:41:11.349057 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69ceee3-67ef-401b-be17-8be64cb49845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:41:11.393256 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-b69ceee3-67ef-401b-be17-8be64cb49845 None None] MFA Rules not processed for user `fe892a32557741aba2cfed60116c959e`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:41:11.394549 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-b69ceee3-67ef-401b-be17-8be64cb49845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:41:11.405692 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa28bda4-2952-474d-bfd7-efc289ec5569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:41:11.408832 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa28bda4-2952-474d-bfd7-efc289ec5569 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:41:11.408832 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa28bda4-2952-474d-bfd7-efc289ec5569 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:41:11.408832 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa28bda4-2952-474d-bfd7-efc289ec5569 admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:41:11.408832 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa28bda4-2952-474d-bfd7-efc289ec5569 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:41:11.410701 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa28bda4-2952-474d-bfd7-efc289ec5569 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:41:11.411685 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:41:11.430392 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1182/2356] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:41:11 2022] GET /identity/v3/roles => generated 1420 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 07:41:11.434743 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa28bda4-2952-474d-bfd7-efc289ec5569 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:41:11.445918 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa28bda4-2952-474d-bfd7-efc289ec5569 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:41:11.451026 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb987f9-15d8-4524-8b07-5da5c371b7eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:41:11.451738 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb987f9-15d8-4524-8b07-5da5c371b7eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:41:11.452023 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb987f9-15d8-4524-8b07-5da5c371b7eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:41:11.452299 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb987f9-15d8-4524-8b07-5da5c371b7eb admin admin] PATH_INFO: `/v3/projects/e5ef5d2e9c5d4352ad39a7d1b1a0170e/users/1fb4689f1f2e48b2a52a1d05f1e98b30/roles/6bbb643e0d8d4bc0a9b27c26edf59216` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:41:11.454678 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb987f9-15d8-4524-8b07-5da5c371b7eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5ef5d2e9c5d4352ad39a7d1b1a0170e, user_id=1fb4689f1f2e48b2a52a1d05f1e98b30, role_id=6bbb643e0d8d4bc0a9b27c26edf59216)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:41:11.457363 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb987f9-15d8-4524-8b07-5da5c371b7eb admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:41:11.469647 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1183/2357] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:41:11 2022] PUT /identity/v3/projects/e5ef5d2e9c5d4352ad39a7d1b1a0170e/users/1fb4689f1f2e48b2a52a1d05f1e98b30/roles/6bbb643e0d8d4bc0a9b27c26edf59216 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:41:11.477120 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1175/2358] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:41:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 07:41:11.491396 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb987f9-15d8-4524-8b07-5da5c371b7eb admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:41:11.493300 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-8cb987f9-15d8-4524-8b07-5da5c371b7eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:41:11.577368 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb987f9-15d8-4524-8b07-5da5c371b7eb admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:41:11.581417 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45360123-a233-4b74-b117-30b7ec52de45 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecf6642b7b84d3c9756bb41ab3349d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:41:11.582064 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45360123-a233-4b74-b117-30b7ec52de45 service cinder] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:41:11.582368 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45360123-a233-4b74-b117-30b7ec52de45 service cinder] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:41:11.582626 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45360123-a233-4b74-b117-30b7ec52de45 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:41:11.586777 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:41:11.639638 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45360123-a233-4b74-b117-30b7ec52de45 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:41:11.641094 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45360123-a233-4b74-b117-30b7ec52de45 service cinder] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:41:11.652082 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1184/2359] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:41:11 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:41:12.529523 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f3708-d8ac-4ff1-91fa-70868bc2032b None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:41:12.530511 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f3708-d8ac-4ff1-91fa-70868bc2032b None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:41:12.531001 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f3708-d8ac-4ff1-91fa-70868bc2032b None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:41:12.534389 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1176/2360] 10.222.0.42 () {54 vars in 921 bytes} [Thu Sep 15 07:41:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:41:12.542472 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d78baf-a726-4ed4-80a9-52538c42e47e None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:41:12.542963 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d78baf-a726-4ed4-80a9-52538c42e47e None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:41:12.543070 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d78baf-a726-4ed4-80a9-52538c42e47e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:41:12.576591 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-a8d78baf-a726-4ed4-80a9-52538c42e47e None None] MFA Rules not processed for user `a080c7ab9b684abca6828067f2572c3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:41:12.577855 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-a8d78baf-a726-4ed4-80a9-52538c42e47e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:41:12.796611 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1185/2361] 10.222.0.42 () {58 vars in 1018 bytes} [Thu Sep 15 07:41:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:41:12.809981 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48f3708-d8ac-4ff1-91fa-70868bc2032b None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:41:12.811734 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-e48f3708-d8ac-4ff1-91fa-70868bc2032b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:41:12.972146 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48f3708-d8ac-4ff1-91fa-70868bc2032b None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:41:12.976832 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba63595-f472-46ac-9098-6d3fb5fb03bf service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37438b032a1f46d3bdf7121ad0cffbc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:41:12.977523 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba63595-f472-46ac-9098-6d3fb5fb03bf service glance] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:41:12.983088 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba63595-f472-46ac-9098-6d3fb5fb03bf service glance] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:41:12.983088 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba63595-f472-46ac-9098-6d3fb5fb03bf service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:41:12.990273 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba63595-f472-46ac-9098-6d3fb5fb03bf service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:41:12.991843 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba63595-f472-46ac-9098-6d3fb5fb03bf service glance] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:41:13.003073 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1177/2362] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:41:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:41:13.275392 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3a1c12-6847-48ff-ae4c-2a43ee1695bf None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:41:13.275727 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3a1c12-6847-48ff-ae4c-2a43ee1695bf None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:41:13.276013 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3a1c12-6847-48ff-ae4c-2a43ee1695bf None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:41:13.277410 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1186/2363] 10.222.0.42 () {54 vars in 921 bytes} [Thu Sep 15 07:41:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:41:13.286145 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2368d2-5772-44d0-8181-b99927b3d95a None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:41:13.286669 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2368d2-5772-44d0-8181-b99927b3d95a None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:41:13.287048 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2368d2-5772-44d0-8181-b99927b3d95a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:41:13.311183 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-fd2368d2-5772-44d0-8181-b99927b3d95a None None] MFA Rules not processed for user `5315996432f44bffa3460b2744c10f0a`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:41:13.312585 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-fd2368d2-5772-44d0-8181-b99927b3d95a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:41:13.428350 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1178/2364] 10.222.0.42 () {58 vars in 1018 bytes} [Thu Sep 15 07:41:13 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:41:13.442055 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3a1c12-6847-48ff-ae4c-2a43ee1695bf None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:41:13.444459 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-5a3a1c12-6847-48ff-ae4c-2a43ee1695bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:41:13.462899 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3a1c12-6847-48ff-ae4c-2a43ee1695bf None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:41:13.480951 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc50720d-ac5b-4e8e-95fe-8195d12f8da7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:41:13.482177 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc50720d-ac5b-4e8e-95fe-8195d12f8da7 service neutron] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:41:13.482177 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc50720d-ac5b-4e8e-95fe-8195d12f8da7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:41:13.482582 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc50720d-ac5b-4e8e-95fe-8195d12f8da7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:41:13.491512 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc50720d-ac5b-4e8e-95fe-8195d12f8da7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:41:13.493038 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc50720d-ac5b-4e8e-95fe-8195d12f8da7 service neutron] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:41:13.504488 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1187/2365] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:41:13 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:41:13.648116 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb2f3d8-d58a-4a71-ad01-98708bf07f97 None None] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:41:13.648658 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb2f3d8-d58a-4a71-ad01-98708bf07f97 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:41:13.648784 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb2f3d8-d58a-4a71-ad01-98708bf07f97 None None] PATH_INFO: `/` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:41:13.650475 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1179/2366] 10.222.0.42 () {54 vars in 921 bytes} [Thu Sep 15 07:41:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 07:41:13.656366 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1b09fd-395c-483a-8009-5b02121f9251 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:41:13.656945 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1b09fd-395c-483a-8009-5b02121f9251 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:41:13.657063 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1b09fd-395c-483a-8009-5b02121f9251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:41:13.680122 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-7f1b09fd-395c-483a-8009-5b02121f9251 None None] MFA Rules not processed for user `a080c7ab9b684abca6828067f2572c3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:41:13.681609 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-7f1b09fd-395c-483a-8009-5b02121f9251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:41:13.694433 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1188/2367] 10.222.0.42 () {58 vars in 1018 bytes} [Thu Sep 15 07:41:13 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 07:41:13.707127 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb2f3d8-d58a-4a71-ad01-98708bf07f97 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:41:13.708935 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-8fb2f3d8-d58a-4a71-ad01-98708bf07f97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:41:13.747394 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb2f3d8-d58a-4a71-ad01-98708bf07f97 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:41:13.770596 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329a7cee-0c45-47c9-bfda-a0f622cefa77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:41:13.771250 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329a7cee-0c45-47c9-bfda-a0f622cefa77 service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:41:13.771580 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329a7cee-0c45-47c9-bfda-a0f622cefa77 service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:41:13.772142 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329a7cee-0c45-47c9-bfda-a0f622cefa77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:41:13.810741 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329a7cee-0c45-47c9-bfda-a0f622cefa77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:41:13.812419 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329a7cee-0c45-47c9-bfda-a0f622cefa77 service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:41:13.834735 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1180/2368] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:41:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:41:19.052239 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62624712-b757-4c98-8c47-61231bc536f8 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:41:19.052705 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62624712-b757-4c98-8c47-61231bc536f8 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:41:19.052796 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62624712-b757-4c98-8c47-61231bc536f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:41:19.089611 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-62624712-b757-4c98-8c47-61231bc536f8 None None] MFA Rules not processed for user `1fb4689f1f2e48b2a52a1d05f1e98b30`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:41:19.091048 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-62624712-b757-4c98-8c47-61231bc536f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:41:19.189695 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1189/2369] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:41:19 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:41:19.205214 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329a7cee-0c45-47c9-bfda-a0f622cefa77 service neutron] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:41:19.206827 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-329a7cee-0c45-47c9-bfda-a0f622cefa77 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:41:19.223653 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329a7cee-0c45-47c9-bfda-a0f622cefa77 service neutron] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:41:19.230055 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17314c3c-cc1c-4c15-a511-cdeed586c469 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a080c7ab9b684abca6828067f2572c3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:41:19.230665 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17314c3c-cc1c-4c15-a511-cdeed586c469 service nova] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:41:19.230958 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17314c3c-cc1c-4c15-a511-cdeed586c469 service nova] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:41:19.231232 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17314c3c-cc1c-4c15-a511-cdeed586c469 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:41:19.300809 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:41:19.306327 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17314c3c-cc1c-4c15-a511-cdeed586c469 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:41:19.308216 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17314c3c-cc1c-4c15-a511-cdeed586c469 service nova] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:41:19.322499 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1181/2370] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:41:19 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:41:21.169525 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58a4b0c-fe7a-40f2-8894-f1d7c9bd2cef None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:41:21.170078 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58a4b0c-fe7a-40f2-8894-f1d7c9bd2cef None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:41:21.172761 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58a4b0c-fe7a-40f2-8894-f1d7c9bd2cef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:41:21.196859 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:41:21.229214 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-f58a4b0c-fe7a-40f2-8894-f1d7c9bd2cef None None] MFA Rules not processed for user `f9c68c4bd9074d8d9b893755e28c0807`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:41:21.229816 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-f58a4b0c-fe7a-40f2-8894-f1d7c9bd2cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:41:21.373439 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1190/2371] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Sep 15 07:41:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:41:21.389145 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17314c3c-cc1c-4c15-a511-cdeed586c469 service nova] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:41:21.391540 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-17314c3c-cc1c-4c15-a511-cdeed586c469 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:41:21.474952 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17314c3c-cc1c-4c15-a511-cdeed586c469 service nova] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:41:21.480578 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf6b6e8-9d1c-4735-8675-f8e6e3042ef3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72b90d35087e4d7a80e9417082a1541f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:41:21.481116 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf6b6e8-9d1c-4735-8675-f8e6e3042ef3 service swift] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:41:21.481401 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf6b6e8-9d1c-4735-8675-f8e6e3042ef3 service swift] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:41:21.481716 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf6b6e8-9d1c-4735-8675-f8e6e3042ef3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:41:21.491097 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf6b6e8-9d1c-4735-8675-f8e6e3042ef3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:41:21.492680 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf6b6e8-9d1c-4735-8675-f8e6e3042ef3 service swift] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:41:21.501622 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1182/2372] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Sep 15 07:41:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 07:41:25.669335 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58a4b0c-fe7a-40f2-8894-f1d7c9bd2cef None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:41:25.777228 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58a4b0c-fe7a-40f2-8894-f1d7c9bd2cef None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:41:25.786714 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1a0d5f-08d7-4262-83c1-ed1482473a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:41:25.787628 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1a0d5f-08d7-4262-83c1-ed1482473a2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:41:25.787959 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1a0d5f-08d7-4262-83c1-ed1482473a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:41:25.788242 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1a0d5f-08d7-4262-83c1-ed1482473a2e admin admin] PATH_INFO: `/v3/users/fe892a32557741aba2cfed60116c959e` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:41:25.791651 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1a0d5f-08d7-4262-83c1-ed1482473a2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe892a32557741aba2cfed60116c959e)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:41:25.793467 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1a0d5f-08d7-4262-83c1-ed1482473a2e admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:41:26.523212 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-eb1a0d5f-08d7-4262-83c1-ed1482473a2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe892a32557741aba2cfed60116c959e'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:41:26.629189 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-eb1a0d5f-08d7-4262-83c1-ed1482473a2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe892a32557741aba2cfed60116c959e'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:41:26.630996 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-eb1a0d5f-08d7-4262-83c1-ed1482473a2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe892a32557741aba2cfed60116c959e'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:41:26.632250 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-eb1a0d5f-08d7-4262-83c1-ed1482473a2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe892a32557741aba2cfed60116c959e'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:41:26.643283 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-eb1a0d5f-08d7-4262-83c1-ed1482473a2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe892a32557741aba2cfed60116c959e'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:41:26.682098 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1191/2373] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:41:25 2022] DELETE /identity/v3/users/fe892a32557741aba2cfed60116c959e => generated 0 bytes in 1013 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 07:41:26.687053 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9ff029-228a-46cf-b62b-a2135182713b None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:41:26.687567 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9ff029-228a-46cf-b62b-a2135182713b None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:41:26.688030 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9ff029-228a-46cf-b62b-a2135182713b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:41:26.715216 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-2d9ff029-228a-46cf-b62b-a2135182713b None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:41:26.716510 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-2d9ff029-228a-46cf-b62b-a2135182713b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:41:26.787844 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1183/2374] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:41:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:41:26.802793 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1a0d5f-08d7-4262-83c1-ed1482473a2e admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:41:26.802793 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-eb1a0d5f-08d7-4262-83c1-ed1482473a2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:41:26.839551 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:41:26.867420 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1a0d5f-08d7-4262-83c1-ed1482473a2e admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:41:26.871165 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7216d1-6582-49cc-9e4c-2c9d65f45699 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:41:26.871780 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7216d1-6582-49cc-9e4c-2c9d65f45699 service neutron] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:41:26.872065 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7216d1-6582-49cc-9e4c-2c9d65f45699 service neutron] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:41:26.872341 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7216d1-6582-49cc-9e4c-2c9d65f45699 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:41:26.882497 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7216d1-6582-49cc-9e4c-2c9d65f45699 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:41:26.883999 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7216d1-6582-49cc-9e4c-2c9d65f45699 service neutron] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:41:26.894500 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1192/2375] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:41:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:41:27.591645 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9ff029-228a-46cf-b62b-a2135182713b None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:41:27.593410 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-2d9ff029-228a-46cf-b62b-a2135182713b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:41:27.630193 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9ff029-228a-46cf-b62b-a2135182713b None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:41:27.645979 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f66af5b-ce5c-4750-aeea-fa454e092ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:41:27.647097 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f66af5b-ce5c-4750-aeea-fa454e092ebe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:41:27.647572 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f66af5b-ce5c-4750-aeea-fa454e092ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:41:27.648117 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f66af5b-ce5c-4750-aeea-fa454e092ebe admin admin] PATH_INFO: `/v3/projects/996835437ec740bfaa4cc5aa8e595b63` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:41:27.649754 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f66af5b-ce5c-4750-aeea-fa454e092ebe admin admin] RBAC: Authorizing `identity:delete_project(project_id=996835437ec740bfaa4cc5aa8e595b63)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:41:27.670963 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f66af5b-ce5c-4750-aeea-fa454e092ebe admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:41:27.889228 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-4f66af5b-ce5c-4750-aeea-fa454e092ebe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '996835437ec740bfaa4cc5aa8e595b63', 'request_id': 'req-4f66af5b-ce5c-4750-aeea-fa454e092ebe'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:41:27.896382 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-4f66af5b-ce5c-4750-aeea-fa454e092ebe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '996835437ec740bfaa4cc5aa8e595b63', 'request_id': 'req-4f66af5b-ce5c-4750-aeea-fa454e092ebe'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:41:27.972546 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-4f66af5b-ce5c-4750-aeea-fa454e092ebe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '996835437ec740bfaa4cc5aa8e595b63', 'request_id': 'req-4f66af5b-ce5c-4750-aeea-fa454e092ebe'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:41:27.985089 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-4f66af5b-ce5c-4750-aeea-fa454e092ebe admin admin] The token cache is being invalidate because project 996835437ec740bfaa4cc5aa8e595b63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:41:27.985387 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-4f66af5b-ce5c-4750-aeea-fa454e092ebe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:41:27.986662 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-4f66af5b-ce5c-4750-aeea-fa454e092ebe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:41:27.988176 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1184/2376] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:41:27 2022] DELETE /identity/v3/projects/996835437ec740bfaa4cc5aa8e595b63 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:41:27.993251 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb17fef-378b-4e13-9ca3-283e5869cac8 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:41:27.993864 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb17fef-378b-4e13-9ca3-283e5869cac8 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:41:27.994024 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb17fef-378b-4e13-9ca3-283e5869cac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:41:28.015202 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-cbb17fef-378b-4e13-9ca3-283e5869cac8 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:41:28.017070 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-cbb17fef-378b-4e13-9ca3-283e5869cac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:41:28.091040 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1193/2377] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:41:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:41:28.100638 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa8287a-4685-435a-a122-db057246a746 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:41:28.100934 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa8287a-4685-435a-a122-db057246a746 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:41:28.101220 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa8287a-4685-435a-a122-db057246a746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:41:28.118958 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:41:28.125738 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-4aa8287a-4685-435a-a122-db057246a746 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:41:28.127642 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-4aa8287a-4685-435a-a122-db057246a746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:41:28.141131 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1185/2378] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:41:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:41:28.143097 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb17fef-378b-4e13-9ca3-283e5869cac8 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:41:28.157034 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb17fef-378b-4e13-9ca3-283e5869cac8 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:41:28.163358 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876afce2-a650-43ab-957e-b77abf8d920d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:41:28.164238 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876afce2-a650-43ab-957e-b77abf8d920d admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:41:28.164611 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876afce2-a650-43ab-957e-b77abf8d920d admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:41:28.164965 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876afce2-a650-43ab-957e-b77abf8d920d admin admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:41:28.165740 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876afce2-a650-43ab-957e-b77abf8d920d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:41:28.166123 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876afce2-a650-43ab-957e-b77abf8d920d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:41:28.167891 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876afce2-a650-43ab-957e-b77abf8d920d admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:41:28.188677 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa8287a-4685-435a-a122-db057246a746 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:41:28.195875 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1194/2379] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 15 07:41:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:41:28.199886 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa8287a-4685-435a-a122-db057246a746 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:41:28.207518 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ada2fa-b815-4009-8651-2dd32c9dacac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:41:28.207518 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ada2fa-b815-4009-8651-2dd32c9dacac admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:41:28.207518 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ada2fa-b815-4009-8651-2dd32c9dacac admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:41:28.207518 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ada2fa-b815-4009-8651-2dd32c9dacac admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:41:28.208059 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ada2fa-b815-4009-8651-2dd32c9dacac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:41:28.208824 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ada2fa-b815-4009-8651-2dd32c9dacac admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:41:28.295451 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1186/2380] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:41:28 2022] POST /identity/v3/projects => generated 373 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:41:28.299935 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876afce2-a650-43ab-957e-b77abf8d920d admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:41:28.324083 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876afce2-a650-43ab-957e-b77abf8d920d admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:41:28.415235 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26e305a-61dd-49a9-9561-9772d636a9f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:41:28.416063 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26e305a-61dd-49a9-9561-9772d636a9f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:41:28.416428 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26e305a-61dd-49a9-9561-9772d636a9f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:41:28.416778 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26e305a-61dd-49a9-9561-9772d636a9f7 admin admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:41:28.417508 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26e305a-61dd-49a9-9561-9772d636a9f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:41:28.419548 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26e305a-61dd-49a9-9561-9772d636a9f7 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:41:28.544928 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1195/2381] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:41:28 2022] POST /identity/v3/users => generated 287 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:41:28.550048 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ada2fa-b815-4009-8651-2dd32c9dacac admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:41:28.562792 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ada2fa-b815-4009-8651-2dd32c9dacac admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:41:28.568807 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df48b3a2-f471-4af4-bf3f-a5a78b781c5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:41:28.569424 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df48b3a2-f471-4af4-bf3f-a5a78b781c5c admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:41:28.569745 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df48b3a2-f471-4af4-bf3f-a5a78b781c5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:41:28.570049 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df48b3a2-f471-4af4-bf3f-a5a78b781c5c admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:41:28.570700 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df48b3a2-f471-4af4-bf3f-a5a78b781c5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:41:28.572350 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df48b3a2-f471-4af4-bf3f-a5a78b781c5c admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:41:28.584005 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1187/2382] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:41:28 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:41:28.587967 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26e305a-61dd-49a9-9561-9772d636a9f7 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:41:28.598765 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26e305a-61dd-49a9-9561-9772d636a9f7 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:41:28.604033 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ec7515-9b8a-4452-a704-ca691c55a168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:41:28.604639 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ec7515-9b8a-4452-a704-ca691c55a168 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:41:28.604917 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ec7515-9b8a-4452-a704-ca691c55a168 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:41:28.605188 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ec7515-9b8a-4452-a704-ca691c55a168 admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:41:28.605951 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ec7515-9b8a-4452-a704-ca691c55a168 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:41:28.607554 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ec7515-9b8a-4452-a704-ca691c55a168 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:41:28.620425 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1196/2383] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:41:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:41:28.624423 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df48b3a2-f471-4af4-bf3f-a5a78b781c5c admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:41:28.635672 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df48b3a2-f471-4af4-bf3f-a5a78b781c5c admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:41:28.640400 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179a8cbb-80b8-40f8-8431-40e313914f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:41:28.641071 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179a8cbb-80b8-40f8-8431-40e313914f16 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:41:28.641350 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179a8cbb-80b8-40f8-8431-40e313914f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:41:28.641822 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179a8cbb-80b8-40f8-8431-40e313914f16 admin admin] PATH_INFO: `/v3/projects/7e20a6e5fded473c8905bffff7775ed3/users/0a1410abac494fd18c859268b528ec0c/roles/053e12af84e943e699460a9d76c2072b` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:41:28.664587 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179a8cbb-80b8-40f8-8431-40e313914f16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e20a6e5fded473c8905bffff7775ed3, user_id=0a1410abac494fd18c859268b528ec0c, role_id=053e12af84e943e699460a9d76c2072b)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:41:28.669639 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179a8cbb-80b8-40f8-8431-40e313914f16 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:41:28.749561 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1188/2384] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:41:28 2022] PUT /identity/v3/projects/7e20a6e5fded473c8905bffff7775ed3/users/0a1410abac494fd18c859268b528ec0c/roles/053e12af84e943e699460a9d76c2072b => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:41:28.755415 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db3874e-6159-48d5-8566-67eef55e56d4 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:41:28.755740 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db3874e-6159-48d5-8566-67eef55e56d4 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:41:28.755968 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db3874e-6159-48d5-8566-67eef55e56d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:41:28.788983 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-4db3874e-6159-48d5-8566-67eef55e56d4 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:41:28.790769 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-4db3874e-6159-48d5-8566-67eef55e56d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:41:28.831031 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:41:28.883402 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1197/2385] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:41:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:41:28.894460 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179a8cbb-80b8-40f8-8431-40e313914f16 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:41:28.896251 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-179a8cbb-80b8-40f8-8431-40e313914f16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:41:28.979890 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179a8cbb-80b8-40f8-8431-40e313914f16 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:41:28.985668 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8363f4-af29-4e11-9450-9a9e346815ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:41:28.986804 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8363f4-af29-4e11-9450-9a9e346815ec service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:41:28.987253 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8363f4-af29-4e11-9450-9a9e346815ec service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:41:28.987687 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8363f4-af29-4e11-9450-9a9e346815ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:41:28.997845 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8363f4-af29-4e11-9450-9a9e346815ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:41:28.999268 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8363f4-af29-4e11-9450-9a9e346815ec service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:41:29.015568 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1189/2386] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:41:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:41:36.063710 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785032c0-ec11-4d63-a7ea-e000b71bcaec None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:41:36.064281 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785032c0-ec11-4d63-a7ea-e000b71bcaec None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:41:36.064281 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785032c0-ec11-4d63-a7ea-e000b71bcaec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:41:36.098199 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-785032c0-ec11-4d63-a7ea-e000b71bcaec None None] MFA Rules not processed for user `0a1410abac494fd18c859268b528ec0c`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:41:36.099692 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-785032c0-ec11-4d63-a7ea-e000b71bcaec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:41:36.195671 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1198/2387] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:41:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:41:36.200157 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8363f4-af29-4e11-9450-9a9e346815ec service neutron] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:41:36.209998 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8363f4-af29-4e11-9450-9a9e346815ec service neutron] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:41:36.217781 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e46da51-6ab2-450a-9961-78719a789fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:41:36.219475 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e46da51-6ab2-450a-9961-78719a789fc1 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:41:36.219790 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e46da51-6ab2-450a-9961-78719a789fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:41:36.220069 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e46da51-6ab2-450a-9961-78719a789fc1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:41:36.220726 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e46da51-6ab2-450a-9961-78719a789fc1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:41:36.222848 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e46da51-6ab2-450a-9961-78719a789fc1 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:41:36.367947 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1190/2388] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:41:36 2022] POST /identity/v3/projects => generated 369 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:41:36.372219 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785032c0-ec11-4d63-a7ea-e000b71bcaec None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:41:36.385819 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785032c0-ec11-4d63-a7ea-e000b71bcaec None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:41:36.466300 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5185ec12-8b73-485d-8133-a109661e80f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:41:36.467483 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5185ec12-8b73-485d-8133-a109661e80f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:41:36.474019 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5185ec12-8b73-485d-8133-a109661e80f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:41:36.476428 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5185ec12-8b73-485d-8133-a109661e80f1 admin admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:41:36.478048 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5185ec12-8b73-485d-8133-a109661e80f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:41:36.483599 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5185ec12-8b73-485d-8133-a109661e80f1 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:41:36.640272 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1199/2389] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:41:36 2022] POST /identity/v3/users => generated 291 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 07:41:36.644541 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e46da51-6ab2-450a-9961-78719a789fc1 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:41:36.655538 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e46da51-6ab2-450a-9961-78719a789fc1 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:41:36.662527 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d810517e-4797-4923-8bbc-665ff78c2939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:41:36.663239 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d810517e-4797-4923-8bbc-665ff78c2939 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:41:36.663523 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d810517e-4797-4923-8bbc-665ff78c2939 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:41:36.663800 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d810517e-4797-4923-8bbc-665ff78c2939 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:41:36.664456 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d810517e-4797-4923-8bbc-665ff78c2939 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:41:36.666098 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d810517e-4797-4923-8bbc-665ff78c2939 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:41:36.679749 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1191/2390] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:41:36 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:41:36.683735 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5185ec12-8b73-485d-8133-a109661e80f1 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:41:36.694836 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5185ec12-8b73-485d-8133-a109661e80f1 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:41:36.699658 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8986786d-67fd-44d7-80bb-e65f30994d6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:41:36.700356 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8986786d-67fd-44d7-80bb-e65f30994d6c admin admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:41:36.700729 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8986786d-67fd-44d7-80bb-e65f30994d6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:41:36.700906 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8986786d-67fd-44d7-80bb-e65f30994d6c admin admin] PATH_INFO: `/v3/projects/c1f22762fda74145b0fd8abb6d133392/users/b07de6d68f1140c8a7e1c893bf2abdd8/roles/6bbb643e0d8d4bc0a9b27c26edf59216` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:41:36.719745 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8986786d-67fd-44d7-80bb-e65f30994d6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1f22762fda74145b0fd8abb6d133392, user_id=b07de6d68f1140c8a7e1c893bf2abdd8, role_id=6bbb643e0d8d4bc0a9b27c26edf59216)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:41:36.723287 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8986786d-67fd-44d7-80bb-e65f30994d6c admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:41:36.797334 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1200/2391] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:41:36 2022] PUT /identity/v3/projects/c1f22762fda74145b0fd8abb6d133392/users/b07de6d68f1140c8a7e1c893bf2abdd8/roles/6bbb643e0d8d4bc0a9b27c26edf59216 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:41:36.801916 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d810517e-4797-4923-8bbc-665ff78c2939 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:41:36.816377 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d810517e-4797-4923-8bbc-665ff78c2939 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:41:36.893316 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b31bff-ef4b-4326-b4bc-3cd32952c0f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:41:36.893975 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b31bff-ef4b-4326-b4bc-3cd32952c0f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:41:36.894275 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b31bff-ef4b-4326-b4bc-3cd32952c0f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:41:36.894527 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b31bff-ef4b-4326-b4bc-3cd32952c0f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:41:36.895167 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b31bff-ef4b-4326-b4bc-3cd32952c0f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:41:36.896780 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b31bff-ef4b-4326-b4bc-3cd32952c0f7 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:41:36.906658 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1192/2392] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:41:36 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:41:36.911735 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8986786d-67fd-44d7-80bb-e65f30994d6c admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:41:36.922170 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8986786d-67fd-44d7-80bb-e65f30994d6c admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:41:36.927058 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fb861a-caf8-4034-9a86-ba0651c26b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:41:36.927718 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fb861a-caf8-4034-9a86-ba0651c26b3e admin admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:41:36.927997 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fb861a-caf8-4034-9a86-ba0651c26b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:41:36.928271 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fb861a-caf8-4034-9a86-ba0651c26b3e admin admin] PATH_INFO: `/v3/projects/c1f22762fda74145b0fd8abb6d133392/users/b07de6d68f1140c8a7e1c893bf2abdd8/roles/6bbb643e0d8d4bc0a9b27c26edf59216` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:41:36.930579 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fb861a-caf8-4034-9a86-ba0651c26b3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1f22762fda74145b0fd8abb6d133392, user_id=b07de6d68f1140c8a7e1c893bf2abdd8, role_id=6bbb643e0d8d4bc0a9b27c26edf59216)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:41:36.933428 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fb861a-caf8-4034-9a86-ba0651c26b3e admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:41:36.944728 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1201/2393] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:41:36 2022] PUT /identity/v3/projects/c1f22762fda74145b0fd8abb6d133392/users/b07de6d68f1140c8a7e1c893bf2abdd8/roles/6bbb643e0d8d4bc0a9b27c26edf59216 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:41:43.121145 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db662167-c03b-4632-b4d0-5bbc7e0db6c7 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:41:43.122043 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db662167-c03b-4632-b4d0-5bbc7e0db6c7 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:41:43.122499 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db662167-c03b-4632-b4d0-5bbc7e0db6c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:41:43.148279 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:41:43.159761 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-db662167-c03b-4632-b4d0-5bbc7e0db6c7 None None] MFA Rules not processed for user `b07de6d68f1140c8a7e1c893bf2abdd8`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:41:43.161353 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-db662167-c03b-4632-b4d0-5bbc7e0db6c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:41:43.235469 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1193/2394] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:41:43 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:41:43.250504 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fb861a-caf8-4034-9a86-ba0651c26b3e admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:41:43.257768 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-70fb861a-caf8-4034-9a86-ba0651c26b3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:41:43.275393 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:41:43.336425 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fb861a-caf8-4034-9a86-ba0651c26b3e admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:41:43.353136 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b9d582-2300-4d12-946a-6bacdb91bc9b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a080c7ab9b684abca6828067f2572c3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:41:43.354341 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b9d582-2300-4d12-946a-6bacdb91bc9b service nova] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:41:43.354633 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b9d582-2300-4d12-946a-6bacdb91bc9b service nova] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:41:43.354935 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b9d582-2300-4d12-946a-6bacdb91bc9b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:41:43.417836 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b9d582-2300-4d12-946a-6bacdb91bc9b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:41:43.419606 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b9d582-2300-4d12-946a-6bacdb91bc9b service nova] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:41:43.430043 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1202/2395] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:41:43 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:41:45.310073 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db662167-c03b-4632-b4d0-5bbc7e0db6c7 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:41:45.311398 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-db662167-c03b-4632-b4d0-5bbc7e0db6c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:41:45.409462 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db662167-c03b-4632-b4d0-5bbc7e0db6c7 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:41:45.409462 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9709536-25c9-4a83-a8dd-5b4b48c7d2df service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd87846b9a134248b28e0521b0a880c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:41:45.421441 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9709536-25c9-4a83-a8dd-5b4b48c7d2df service placement] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:41:45.423701 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9709536-25c9-4a83-a8dd-5b4b48c7d2df service placement] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:41:45.423987 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9709536-25c9-4a83-a8dd-5b4b48c7d2df service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:41:45.426958 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-d9709536-25c9-4a83-a8dd-5b4b48c7d2df service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:41:45.443657 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9709536-25c9-4a83-a8dd-5b4b48c7d2df service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:41:45.445457 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9709536-25c9-4a83-a8dd-5b4b48c7d2df service placement] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:41:45.459263 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1194/2396] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:41:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:41:46.606597 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abd271a-7371-4968-8a2f-0f53fb7e2ba5 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:41:46.607176 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abd271a-7371-4968-8a2f-0f53fb7e2ba5 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:41:46.607275 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abd271a-7371-4968-8a2f-0f53fb7e2ba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:41:46.628894 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-2abd271a-7371-4968-8a2f-0f53fb7e2ba5 None None] MFA Rules not processed for user `0a1410abac494fd18c859268b528ec0c`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:41:46.630509 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-2abd271a-7371-4968-8a2f-0f53fb7e2ba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:41:46.640102 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1203/2397] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:41:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:41:48.367370 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aa9b21-2e4d-49f4-8314-e7826db8ed13 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:41:48.367850 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aa9b21-2e4d-49f4-8314-e7826db8ed13 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:41:48.368077 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aa9b21-2e4d-49f4-8314-e7826db8ed13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:41:48.388456 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-01aa9b21-2e4d-49f4-8314-e7826db8ed13 None None] MFA Rules not processed for user `ebd85e42855448268c89a200f4392316`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:41:48.389755 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-01aa9b21-2e4d-49f4-8314-e7826db8ed13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:41:48.447811 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1195/2398] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:41:48 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:41:56.621686 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=51 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:41:58.440571 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=87 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:42:01.188454 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a787f2-f5c3-4b85-816e-63888742b5ab None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:42:01.190762 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a787f2-f5c3-4b85-816e-63888742b5ab None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:42:01.191335 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a787f2-f5c3-4b85-816e-63888742b5ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:42:01.237356 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-92a787f2-f5c3-4b85-816e-63888742b5ab None None] MFA Rules not processed for user `f9c68c4bd9074d8d9b893755e28c0807`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:42:01.240030 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-92a787f2-f5c3-4b85-816e-63888742b5ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:42:01.425438 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1204/2399] 10.222.0.42 () {58 vars in 967 bytes} [Thu Sep 15 07:42:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:42:01.455251 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aa9b21-2e4d-49f4-8314-e7826db8ed13 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:42:01.462005 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-01aa9b21-2e4d-49f4-8314-e7826db8ed13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:42:01.626715 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aa9b21-2e4d-49f4-8314-e7826db8ed13 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:42:01.640274 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886e1ce8-744c-4b78-9718-bc30242dba42 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72b90d35087e4d7a80e9417082a1541f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:42:01.640274 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886e1ce8-744c-4b78-9718-bc30242dba42 service swift] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:42:01.640274 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886e1ce8-744c-4b78-9718-bc30242dba42 service swift] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:42:01.640274 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886e1ce8-744c-4b78-9718-bc30242dba42 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:42:01.648913 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886e1ce8-744c-4b78-9718-bc30242dba42 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:42:01.651399 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886e1ce8-744c-4b78-9718-bc30242dba42 service swift] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:42:01.672436 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1196/2400] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Sep 15 07:42:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 07:42:11.408144 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=25 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:42:11.599224 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd0d904-56d1-4f0c-9789-8ad700a6db06 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:42:11.599591 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd0d904-56d1-4f0c-9789-8ad700a6db06 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:42:11.599937 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd0d904-56d1-4f0c-9789-8ad700a6db06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:42:11.619665 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-2fd0d904-56d1-4f0c-9789-8ad700a6db06 None None] MFA Rules not processed for user `21de3a85b2df4af2833a24ab02f477c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:42:11.621597 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-2fd0d904-56d1-4f0c-9789-8ad700a6db06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:42:11.664632 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=32 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:42:11.678533 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1205/2401] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:42:11 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:42:21.671711 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=25 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:42:22.246922 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34575e01-c963-4933-8c6e-fee807dd57e9 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:42:22.247372 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34575e01-c963-4933-8c6e-fee807dd57e9 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:42:22.247496 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34575e01-c963-4933-8c6e-fee807dd57e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:42:22.266230 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-34575e01-c963-4933-8c6e-fee807dd57e9 None None] MFA Rules not processed for user `5e0c8d7aa4db455a94631becf212866f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:42:22.267580 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-34575e01-c963-4933-8c6e-fee807dd57e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:42:22.326455 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1197/2402] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:42:22 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:42:25.979994 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f7860e-1437-4681-8621-fa928edaed87 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:42:25.980429 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f7860e-1437-4681-8621-fa928edaed87 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:42:25.980682 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f7860e-1437-4681-8621-fa928edaed87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:42:26.002433 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-b2f7860e-1437-4681-8621-fa928edaed87 None None] MFA Rules not processed for user `ae7e3a9a28134a28a3f8f30d4b0fcb8c`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:42:26.003860 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-b2f7860e-1437-4681-8621-fa928edaed87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:42:26.074886 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1206/2403] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:42:25 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:42:28.830536 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34575e01-c963-4933-8c6e-fee807dd57e9 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:42:28.916999 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34575e01-c963-4933-8c6e-fee807dd57e9 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:42:28.922709 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58a6ee6-08ef-45cc-9eb2-a743be305331 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:42:28.923469 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58a6ee6-08ef-45cc-9eb2-a743be305331 service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:42:28.924665 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58a6ee6-08ef-45cc-9eb2-a743be305331 service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:42:28.925279 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58a6ee6-08ef-45cc-9eb2-a743be305331 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:42:28.928096 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-e58a6ee6-08ef-45cc-9eb2-a743be305331 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:42:29.048947 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58a6ee6-08ef-45cc-9eb2-a743be305331 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:42:29.050425 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58a6ee6-08ef-45cc-9eb2-a743be305331 service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:42:29.063802 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1198/2404] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:42:28 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:42:36.063693 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=25 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:42:37.055847 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f7860e-1437-4681-8621-fa928edaed87 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:42:37.066042 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f7860e-1437-4681-8621-fa928edaed87 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:42:37.069839 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6170ec-e657-4067-accf-d0b89559798d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:42:37.070457 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6170ec-e657-4067-accf-d0b89559798d service neutron] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:42:37.070741 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6170ec-e657-4067-accf-d0b89559798d service neutron] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:42:37.071016 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6170ec-e657-4067-accf-d0b89559798d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:42:37.081186 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6170ec-e657-4067-accf-d0b89559798d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:42:37.082682 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6170ec-e657-4067-accf-d0b89559798d service neutron] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:42:37.092862 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1207/2405] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:42:37 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:42:39.056232 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=75 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:42:47.088165 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=6 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:43:09.006770 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f86d8-8b8d-4fd6-a62f-98564cbb8dfd None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:43:09.011662 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f86d8-8b8d-4fd6-a62f-98564cbb8dfd None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:43:09.011662 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f86d8-8b8d-4fd6-a62f-98564cbb8dfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:43:09.027817 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-ff7f86d8-8b8d-4fd6-a62f-98564cbb8dfd None None] MFA Rules not processed for user `56c2f854cf074b1ba2571766b7f2fc51`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:43:09.029044 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-ff7f86d8-8b8d-4fd6-a62f-98564cbb8dfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:43:09.083980 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1199/2406] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:43:09 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:43:19.076708 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=25 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:43:37.869299 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6170ec-e657-4067-accf-d0b89559798d service neutron] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:43:37.870740 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-da6170ec-e657-4067-accf-d0b89559798d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:43:37.927472 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6170ec-e657-4067-accf-d0b89559798d service neutron] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:43:37.931067 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9973934a-6f2b-451d-a1fe-8676e0256311 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecf6642b7b84d3c9756bb41ab3349d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:43:37.931663 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9973934a-6f2b-451d-a1fe-8676e0256311 service cinder] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:43:37.931975 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9973934a-6f2b-451d-a1fe-8676e0256311 service cinder] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:43:37.932241 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9973934a-6f2b-451d-a1fe-8676e0256311 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:43:37.933940 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-9973934a-6f2b-451d-a1fe-8676e0256311 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:43:37.951135 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9973934a-6f2b-451d-a1fe-8676e0256311 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:43:37.951135 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9973934a-6f2b-451d-a1fe-8676e0256311 service cinder] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:43:37.960963 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1208/2407] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:43:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:43:38.373153 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7f86d8-8b8d-4fd6-a62f-98564cbb8dfd None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:43:38.383870 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7f86d8-8b8d-4fd6-a62f-98564cbb8dfd None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:43:38.389385 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7b76b7-a8cf-4171-a177-2941c250ab16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:43:38.390108 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7b76b7-a8cf-4171-a177-2941c250ab16 service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:43:38.390492 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7b76b7-a8cf-4171-a177-2941c250ab16 service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:43:38.390768 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7b76b7-a8cf-4171-a177-2941c250ab16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:43:38.392508 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-3d7b76b7-a8cf-4171-a177-2941c250ab16 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:43:38.406712 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7b76b7-a8cf-4171-a177-2941c250ab16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:43:38.410349 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7b76b7-a8cf-4171-a177-2941c250ab16 service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:43:38.423504 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1200/2408] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:43:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:43:41.429446 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9973934a-6f2b-451d-a1fe-8676e0256311 service cinder] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:43:41.431455 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-9973934a-6f2b-451d-a1fe-8676e0256311 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:43:41.496936 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9973934a-6f2b-451d-a1fe-8676e0256311 service cinder] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:43:41.501385 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69de36aa-63f6-4124-95c1-fb2cb7ccee1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:43:41.502052 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de36aa-63f6-4124-95c1-fb2cb7ccee1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:43:41.502337 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de36aa-63f6-4124-95c1-fb2cb7ccee1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:43:41.502615 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de36aa-63f6-4124-95c1-fb2cb7ccee1d admin admin] PATH_INFO: `/v3/users/ae7e3a9a28134a28a3f8f30d4b0fcb8c` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:43:41.503750 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69de36aa-63f6-4124-95c1-fb2cb7ccee1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae7e3a9a28134a28a3f8f30d4b0fcb8c)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:43:41.505721 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69de36aa-63f6-4124-95c1-fb2cb7ccee1d admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:43:41.674709 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-69de36aa-63f6-4124-95c1-fb2cb7ccee1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae7e3a9a28134a28a3f8f30d4b0fcb8c'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:43:41.753046 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-69de36aa-63f6-4124-95c1-fb2cb7ccee1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae7e3a9a28134a28a3f8f30d4b0fcb8c'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:43:41.753605 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-69de36aa-63f6-4124-95c1-fb2cb7ccee1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae7e3a9a28134a28a3f8f30d4b0fcb8c'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:43:41.754220 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-69de36aa-63f6-4124-95c1-fb2cb7ccee1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae7e3a9a28134a28a3f8f30d4b0fcb8c'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:43:41.762800 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-69de36aa-63f6-4124-95c1-fb2cb7ccee1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae7e3a9a28134a28a3f8f30d4b0fcb8c'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:43:41.780764 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1209/2409] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:43:41 2022] DELETE /identity/v3/users/ae7e3a9a28134a28a3f8f30d4b0fcb8c => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 07:43:42.023189 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7b76b7-a8cf-4171-a177-2941c250ab16 service neutron] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:43:42.035110 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7b76b7-a8cf-4171-a177-2941c250ab16 service neutron] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:43:42.105975 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db80cdd-c263-44db-a935-bdf62376ef13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:43:42.106619 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db80cdd-c263-44db-a935-bdf62376ef13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:43:42.106906 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db80cdd-c263-44db-a935-bdf62376ef13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:43:42.107186 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db80cdd-c263-44db-a935-bdf62376ef13 admin admin] PATH_INFO: `/v3/projects/66cb9c7c7eb24e30bf89b3680c713472` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:43:42.108341 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db80cdd-c263-44db-a935-bdf62376ef13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66cb9c7c7eb24e30bf89b3680c713472)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:43:42.110309 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db80cdd-c263-44db-a935-bdf62376ef13 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:43:42.247251 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-5db80cdd-c263-44db-a935-bdf62376ef13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66cb9c7c7eb24e30bf89b3680c713472', 'request_id': 'req-5db80cdd-c263-44db-a935-bdf62376ef13'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:43:42.253820 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-5db80cdd-c263-44db-a935-bdf62376ef13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66cb9c7c7eb24e30bf89b3680c713472', 'request_id': 'req-5db80cdd-c263-44db-a935-bdf62376ef13'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:43:42.313588 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-5db80cdd-c263-44db-a935-bdf62376ef13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66cb9c7c7eb24e30bf89b3680c713472', 'request_id': 'req-5db80cdd-c263-44db-a935-bdf62376ef13'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:43:42.325274 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-5db80cdd-c263-44db-a935-bdf62376ef13 admin admin] The token cache is being invalidate because project 66cb9c7c7eb24e30bf89b3680c713472 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:43:42.325568 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-5db80cdd-c263-44db-a935-bdf62376ef13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:43:42.326845 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-5db80cdd-c263-44db-a935-bdf62376ef13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:43:42.328386 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1201/2410] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:43:42 2022] DELETE /identity/v3/projects/66cb9c7c7eb24e30bf89b3680c713472 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:43:48.201255 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69de36aa-63f6-4124-95c1-fb2cb7ccee1d admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:43:48.202686 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-69de36aa-63f6-4124-95c1-fb2cb7ccee1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:43:48.229299 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:43:48.260027 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69de36aa-63f6-4124-95c1-fb2cb7ccee1d admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:43:48.308565 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c4a1ab-811d-4c44-8f99-344d6e79bc97 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecf6642b7b84d3c9756bb41ab3349d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:43:48.309173 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c4a1ab-811d-4c44-8f99-344d6e79bc97 service cinder] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:43:48.309449 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c4a1ab-811d-4c44-8f99-344d6e79bc97 service cinder] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:43:48.309768 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c4a1ab-811d-4c44-8f99-344d6e79bc97 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:43:48.312231 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-e1c4a1ab-811d-4c44-8f99-344d6e79bc97 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:43:48.370358 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c4a1ab-811d-4c44-8f99-344d6e79bc97 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:43:48.371817 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c4a1ab-811d-4c44-8f99-344d6e79bc97 service cinder] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:43:48.380542 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1210/2411] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:43:48 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:43:52.322127 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=57,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:43:55.352825 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db80cdd-c263-44db-a935-bdf62376ef13 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:43:55.354717 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-5db80cdd-c263-44db-a935-bdf62376ef13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:43:55.418712 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db80cdd-c263-44db-a935-bdf62376ef13 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:43:55.422544 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a259f03-7a3f-45f8-8362-f6acc7a19e37 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:43:55.423372 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a259f03-7a3f-45f8-8362-f6acc7a19e37 service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:43:55.423716 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a259f03-7a3f-45f8-8362-f6acc7a19e37 service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:43:55.424078 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a259f03-7a3f-45f8-8362-f6acc7a19e37 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:43:55.425937 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-0a259f03-7a3f-45f8-8362-f6acc7a19e37 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:43:55.436198 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a259f03-7a3f-45f8-8362-f6acc7a19e37 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:43:55.437847 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a259f03-7a3f-45f8-8362-f6acc7a19e37 service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:43:55.446868 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1202/2412] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:43:55 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:43:57.538986 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8f4be0-ec15-4412-8312-cdd7a7ad864d None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:43:57.539485 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8f4be0-ec15-4412-8312-cdd7a7ad864d None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:43:57.539600 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8f4be0-ec15-4412-8312-cdd7a7ad864d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:43:57.585630 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-ee8f4be0-ec15-4412-8312-cdd7a7ad864d None None] MFA Rules not processed for user `21de3a85b2df4af2833a24ab02f477c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:43:57.586839 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-ee8f4be0-ec15-4412-8312-cdd7a7ad864d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:43:57.658861 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1211/2413] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:43:57 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:44:00.340446 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5429549a-e1c5-475d-a90f-f4cc80d4fc53 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:44:00.341011 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5429549a-e1c5-475d-a90f-f4cc80d4fc53 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:44:00.341114 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5429549a-e1c5-475d-a90f-f4cc80d4fc53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:44:00.360316 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-5429549a-e1c5-475d-a90f-f4cc80d4fc53 None None] MFA Rules not processed for user `5e0c8d7aa4db455a94631becf212866f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:44:00.361505 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-5429549a-e1c5-475d-a90f-f4cc80d4fc53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:44:00.368704 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1203/2414] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:44:00 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:44:01.023703 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8f4be0-ec15-4412-8312-cdd7a7ad864d None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:44:01.025150 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-ee8f4be0-ec15-4412-8312-cdd7a7ad864d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:44:01.037739 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8f4be0-ec15-4412-8312-cdd7a7ad864d None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:44:01.041360 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c304c6d6-21c5-4d9b-9aad-c009570743c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:44:01.042005 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c304c6d6-21c5-4d9b-9aad-c009570743c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:44:01.042288 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c304c6d6-21c5-4d9b-9aad-c009570743c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:44:01.042562 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c304c6d6-21c5-4d9b-9aad-c009570743c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:44:01.044325 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-c304c6d6-21c5-4d9b-9aad-c009570743c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:44:01.055670 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c304c6d6-21c5-4d9b-9aad-c009570743c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:44:01.057115 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c304c6d6-21c5-4d9b-9aad-c009570743c1 service neutron] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:44:01.065692 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1212/2415] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:44:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:44:01.669277 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5429549a-e1c5-475d-a90f-f4cc80d4fc53 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:44:01.670857 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-5429549a-e1c5-475d-a90f-f4cc80d4fc53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:44:01.743771 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5429549a-e1c5-475d-a90f-f4cc80d4fc53 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:44:01.748749 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4292c86f-9317-4fd7-b2da-cdc74b1f7272 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd87846b9a134248b28e0521b0a880c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:44:01.749356 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4292c86f-9317-4fd7-b2da-cdc74b1f7272 service placement] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:44:01.749665 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4292c86f-9317-4fd7-b2da-cdc74b1f7272 service placement] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:44:01.749971 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4292c86f-9317-4fd7-b2da-cdc74b1f7272 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:44:01.751721 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-4292c86f-9317-4fd7-b2da-cdc74b1f7272 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:44:01.765479 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4292c86f-9317-4fd7-b2da-cdc74b1f7272 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:44:01.767084 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4292c86f-9317-4fd7-b2da-cdc74b1f7272 service placement] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:44:01.777479 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1204/2416] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:44:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:44:02.066611 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c304c6d6-21c5-4d9b-9aad-c009570743c1 service neutron] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:44:02.070190 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-c304c6d6-21c5-4d9b-9aad-c009570743c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:44:02.108201 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:44:02.169173 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c304c6d6-21c5-4d9b-9aad-c009570743c1 service neutron] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:44:02.175370 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a59d5a-2313-4248-a104-4627c3cfcef5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a080c7ab9b684abca6828067f2572c3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:44:02.176926 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a59d5a-2313-4248-a104-4627c3cfcef5 service nova] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:44:02.176926 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a59d5a-2313-4248-a104-4627c3cfcef5 service nova] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:44:02.176926 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a59d5a-2313-4248-a104-4627c3cfcef5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:44:02.178453 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-d1a59d5a-2313-4248-a104-4627c3cfcef5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:44:02.191676 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a59d5a-2313-4248-a104-4627c3cfcef5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:44:02.193332 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a59d5a-2313-4248-a104-4627c3cfcef5 service nova] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:44:02.205313 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1213/2417] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:44:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:44:03.547999 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4292c86f-9317-4fd7-b2da-cdc74b1f7272 service placement] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:44:03.560254 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4292c86f-9317-4fd7-b2da-cdc74b1f7272 service placement] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:44:03.565993 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0645c653-5269-43d5-ab39-dd7a78302070 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a080c7ab9b684abca6828067f2572c3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:44:03.566567 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0645c653-5269-43d5-ab39-dd7a78302070 service nova] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:44:03.566858 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0645c653-5269-43d5-ab39-dd7a78302070 service nova] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:44:03.567136 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0645c653-5269-43d5-ab39-dd7a78302070 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:44:03.568909 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-0645c653-5269-43d5-ab39-dd7a78302070 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:44:03.628906 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0645c653-5269-43d5-ab39-dd7a78302070 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:44:03.630517 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0645c653-5269-43d5-ab39-dd7a78302070 service nova] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:44:03.639503 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1205/2418] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:44:03 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:44:12.200841 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=23 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:44:12.581767 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a59d5a-2313-4248-a104-4627c3cfcef5 service nova] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:44:12.583177 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-d1a59d5a-2313-4248-a104-4627c3cfcef5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:44:12.602682 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a59d5a-2313-4248-a104-4627c3cfcef5 service nova] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:44:12.603340 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f305b91a-e0dc-41bd-bc4d-9324348835ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a080c7ab9b684abca6828067f2572c3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:44:12.603941 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f305b91a-e0dc-41bd-bc4d-9324348835ff service nova] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:44:12.604222 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f305b91a-e0dc-41bd-bc4d-9324348835ff service nova] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:44:12.604515 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f305b91a-e0dc-41bd-bc4d-9324348835ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:44:12.606215 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-f305b91a-e0dc-41bd-bc4d-9324348835ff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:44:12.619520 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f305b91a-e0dc-41bd-bc4d-9324348835ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:44:12.621114 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f305b91a-e0dc-41bd-bc4d-9324348835ff service nova] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:44:12.633601 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1214/2419] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:44:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:44:13.635560 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=89 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:44:16.763753 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0645c653-5269-43d5-ab39-dd7a78302070 service nova] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:44:16.775235 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0645c653-5269-43d5-ab39-dd7a78302070 service nova] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:44:16.780858 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd42e3ad-99c6-47fa-b9dc-5940e7a8dbeb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a080c7ab9b684abca6828067f2572c3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:44:16.782101 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd42e3ad-99c6-47fa-b9dc-5940e7a8dbeb service nova] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:44:16.782340 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd42e3ad-99c6-47fa-b9dc-5940e7a8dbeb service nova] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:44:16.782622 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd42e3ad-99c6-47fa-b9dc-5940e7a8dbeb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:44:16.784416 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-fd42e3ad-99c6-47fa-b9dc-5940e7a8dbeb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:44:16.796670 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd42e3ad-99c6-47fa-b9dc-5940e7a8dbeb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:44:16.798953 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd42e3ad-99c6-47fa-b9dc-5940e7a8dbeb service nova] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:44:16.810051 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1206/2420] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:44:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:44:22.628961 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=6 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:44:24.730530 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5a69ad-4417-4b27-b37d-5ccaa80b0251 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:44:24.731162 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5a69ad-4417-4b27-b37d-5ccaa80b0251 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:44:24.731162 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5a69ad-4417-4b27-b37d-5ccaa80b0251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:44:24.756296 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-1a5a69ad-4417-4b27-b37d-5ccaa80b0251 None None] MFA Rules not processed for user `f9c68c4bd9074d8d9b893755e28c0807`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:44:24.757714 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-1a5a69ad-4417-4b27-b37d-5ccaa80b0251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:44:24.817082 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1215/2421] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Sep 15 07:44:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:44:24.991803 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd42e3ad-99c6-47fa-b9dc-5940e7a8dbeb service nova] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:44:24.993248 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-fd42e3ad-99c6-47fa-b9dc-5940e7a8dbeb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:44:25.069235 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd42e3ad-99c6-47fa-b9dc-5940e7a8dbeb service nova] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:44:25.074146 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0ebaed-a27b-4b3f-91b4-7bc6cb712f28 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72b90d35087e4d7a80e9417082a1541f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:44:25.074752 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0ebaed-a27b-4b3f-91b4-7bc6cb712f28 service swift] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:44:25.075036 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0ebaed-a27b-4b3f-91b4-7bc6cb712f28 service swift] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:44:25.075347 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0ebaed-a27b-4b3f-91b4-7bc6cb712f28 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:44:25.083919 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0ebaed-a27b-4b3f-91b4-7bc6cb712f28 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:44:25.085491 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0ebaed-a27b-4b3f-91b4-7bc6cb712f28 service swift] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:44:25.093386 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1207/2422] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Sep 15 07:44:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 07:44:34.809468 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=25 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:44:35.090203 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=38 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:44:47.257206 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c1cc1f-4a9e-4b5c-8d54-43d49be6cfac None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:44:47.257783 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c1cc1f-4a9e-4b5c-8d54-43d49be6cfac None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:44:47.257942 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c1cc1f-4a9e-4b5c-8d54-43d49be6cfac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:44:47.277451 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-d6c1cc1f-4a9e-4b5c-8d54-43d49be6cfac None None] MFA Rules not processed for user `f9c68c4bd9074d8d9b893755e28c0807`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:44:47.278685 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-d6c1cc1f-4a9e-4b5c-8d54-43d49be6cfac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:44:47.285760 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1216/2423] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Sep 15 07:44:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:44:47.298732 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0ebaed-a27b-4b3f-91b4-7bc6cb712f28 service swift] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:44:47.309522 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0ebaed-a27b-4b3f-91b4-7bc6cb712f28 service swift] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:44:47.314544 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bba2ac-f42e-4496-a205-0de4b6dda4d4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72b90d35087e4d7a80e9417082a1541f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:44:47.315143 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bba2ac-f42e-4496-a205-0de4b6dda4d4 service swift] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:44:47.315428 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bba2ac-f42e-4496-a205-0de4b6dda4d4 service swift] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:44:47.315705 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bba2ac-f42e-4496-a205-0de4b6dda4d4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:44:47.323666 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bba2ac-f42e-4496-a205-0de4b6dda4d4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:44:47.325248 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bba2ac-f42e-4496-a205-0de4b6dda4d4 service swift] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:44:47.337259 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1208/2424] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Sep 15 07:44:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 07:44:57.271634 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=3 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:44:57.333881 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=6 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:45:05.944541 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc1d6b0-a257-445c-8957-2a86cabff4eb None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:05.948994 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc1d6b0-a257-445c-8957-2a86cabff4eb None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:05.948994 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc1d6b0-a257-445c-8957-2a86cabff4eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:05.965713 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-ffc1d6b0-a257-445c-8957-2a86cabff4eb None None] MFA Rules not processed for user `f9c68c4bd9074d8d9b893755e28c0807`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:45:05.966970 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-ffc1d6b0-a257-445c-8957-2a86cabff4eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:45:05.974078 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1217/2425] 10.222.0.42 () {58 vars in 967 bytes} [Thu Sep 15 07:45:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:45:05.986662 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bba2ac-f42e-4496-a205-0de4b6dda4d4 service swift] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:06.001037 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bba2ac-f42e-4496-a205-0de4b6dda4d4 service swift] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:06.006529 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fda4fc-de79-47e1-ab3b-4d09a8a21411 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72b90d35087e4d7a80e9417082a1541f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:06.007140 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fda4fc-de79-47e1-ab3b-4d09a8a21411 service swift] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:06.007423 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fda4fc-de79-47e1-ab3b-4d09a8a21411 service swift] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:06.008392 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fda4fc-de79-47e1-ab3b-4d09a8a21411 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:06.017481 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fda4fc-de79-47e1-ab3b-4d09a8a21411 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:06.019176 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fda4fc-de79-47e1-ab3b-4d09a8a21411 service swift] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:06.027702 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1209/2426] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Sep 15 07:45:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 07:45:08.378662 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be84742-0584-48b0-bb90-6db1f4abbd24 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:08.379137 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be84742-0584-48b0-bb90-6db1f4abbd24 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:08.379234 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be84742-0584-48b0-bb90-6db1f4abbd24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:08.399111 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-6be84742-0584-48b0-bb90-6db1f4abbd24 None None] MFA Rules not processed for user `ebd85e42855448268c89a200f4392316`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:45:08.400335 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-6be84742-0584-48b0-bb90-6db1f4abbd24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:45:08.456848 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1218/2427] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:45:08 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:45:09.078945 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fda4fc-de79-47e1-ab3b-4d09a8a21411 service swift] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:09.091378 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fda4fc-de79-47e1-ab3b-4d09a8a21411 service swift] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:09.097678 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33efa43f-f88a-4794-88f3-7e188a2791a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a080c7ab9b684abca6828067f2572c3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:09.098291 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33efa43f-f88a-4794-88f3-7e188a2791a9 service nova] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:09.098566 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33efa43f-f88a-4794-88f3-7e188a2791a9 service nova] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:09.098858 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33efa43f-f88a-4794-88f3-7e188a2791a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:09.100700 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-33efa43f-f88a-4794-88f3-7e188a2791a9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:45:09.125301 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33efa43f-f88a-4794-88f3-7e188a2791a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:09.126982 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33efa43f-f88a-4794-88f3-7e188a2791a9 service nova] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:09.137544 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1210/2428] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:45:09 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:45:09.186871 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be84742-0584-48b0-bb90-6db1f4abbd24 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:09.196663 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be84742-0584-48b0-bb90-6db1f4abbd24 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:09.200914 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5907533-013e-474d-9c64-31354d2eda77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:09.201523 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5907533-013e-474d-9c64-31354d2eda77 service neutron] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:09.201841 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5907533-013e-474d-9c64-31354d2eda77 service neutron] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:09.202118 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5907533-013e-474d-9c64-31354d2eda77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:09.203865 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-b5907533-013e-474d-9c64-31354d2eda77 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:45:09.280083 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5907533-013e-474d-9c64-31354d2eda77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:09.281525 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5907533-013e-474d-9c64-31354d2eda77 service neutron] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:09.291479 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1219/2429] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:45:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:45:13.324701 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33efa43f-f88a-4794-88f3-7e188a2791a9 service nova] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:13.348838 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33efa43f-f88a-4794-88f3-7e188a2791a9 service nova] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:13.357650 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7eab821-9a6b-4f80-ba02-748c2bdf64a4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd87846b9a134248b28e0521b0a880c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:13.358393 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eab821-9a6b-4f80-ba02-748c2bdf64a4 service placement] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:13.358750 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eab821-9a6b-4f80-ba02-748c2bdf64a4 service placement] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:13.359568 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eab821-9a6b-4f80-ba02-748c2bdf64a4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:13.361609 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-d7eab821-9a6b-4f80-ba02-748c2bdf64a4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:45:13.376704 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7eab821-9a6b-4f80-ba02-748c2bdf64a4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:13.378451 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7eab821-9a6b-4f80-ba02-748c2bdf64a4 service placement] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:13.391982 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1211/2430] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:45:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:45:13.516048 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5907533-013e-474d-9c64-31354d2eda77 service neutron] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:13.517665 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-b5907533-013e-474d-9c64-31354d2eda77 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:45:13.534348 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5907533-013e-474d-9c64-31354d2eda77 service neutron] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:13.547446 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db62ef8f-5571-4a5a-8c56-4d0a8fc83daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:13.548124 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db62ef8f-5571-4a5a-8c56-4d0a8fc83daf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:13.548408 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db62ef8f-5571-4a5a-8c56-4d0a8fc83daf admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:13.548680 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db62ef8f-5571-4a5a-8c56-4d0a8fc83daf admin admin] PATH_INFO: `/v3/users/21de3a85b2df4af2833a24ab02f477c8` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:13.550073 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db62ef8f-5571-4a5a-8c56-4d0a8fc83daf admin admin] RBAC: Authorizing `identity:delete_user(user_id=21de3a85b2df4af2833a24ab02f477c8)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:13.552273 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db62ef8f-5571-4a5a-8c56-4d0a8fc83daf admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:13.774264 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-db62ef8f-5571-4a5a-8c56-4d0a8fc83daf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21de3a85b2df4af2833a24ab02f477c8'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:13.873830 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-db62ef8f-5571-4a5a-8c56-4d0a8fc83daf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21de3a85b2df4af2833a24ab02f477c8'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:13.874539 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-db62ef8f-5571-4a5a-8c56-4d0a8fc83daf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21de3a85b2df4af2833a24ab02f477c8'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:13.875173 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-db62ef8f-5571-4a5a-8c56-4d0a8fc83daf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21de3a85b2df4af2833a24ab02f477c8'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:13.883884 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-db62ef8f-5571-4a5a-8c56-4d0a8fc83daf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21de3a85b2df4af2833a24ab02f477c8'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:13.909126 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1220/2431] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:45:13 2022] DELETE /identity/v3/users/21de3a85b2df4af2833a24ab02f477c8 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:45:14.269893 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7eab821-9a6b-4f80-ba02-748c2bdf64a4 service placement] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:14.296039 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7eab821-9a6b-4f80-ba02-748c2bdf64a4 service placement] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:14.371510 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2563b068-f978-45db-b018-6a5c3c1f2333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:14.373129 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2563b068-f978-45db-b018-6a5c3c1f2333 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:14.378050 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2563b068-f978-45db-b018-6a5c3c1f2333 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:14.378050 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2563b068-f978-45db-b018-6a5c3c1f2333 admin admin] PATH_INFO: `/v3/projects/cdc69db806d24d1f8eb18aadaaafabcb` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:14.378050 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2563b068-f978-45db-b018-6a5c3c1f2333 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cdc69db806d24d1f8eb18aadaaafabcb)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:14.378738 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2563b068-f978-45db-b018-6a5c3c1f2333 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:14.492575 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-2563b068-f978-45db-b018-6a5c3c1f2333 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cdc69db806d24d1f8eb18aadaaafabcb', 'request_id': 'req-2563b068-f978-45db-b018-6a5c3c1f2333'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:14.506983 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-2563b068-f978-45db-b018-6a5c3c1f2333 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cdc69db806d24d1f8eb18aadaaafabcb', 'request_id': 'req-2563b068-f978-45db-b018-6a5c3c1f2333'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:14.571369 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-2563b068-f978-45db-b018-6a5c3c1f2333 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cdc69db806d24d1f8eb18aadaaafabcb', 'request_id': 'req-2563b068-f978-45db-b018-6a5c3c1f2333'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:14.584695 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-2563b068-f978-45db-b018-6a5c3c1f2333 admin admin] The token cache is being invalidate because project cdc69db806d24d1f8eb18aadaaafabcb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:45:14.585368 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-2563b068-f978-45db-b018-6a5c3c1f2333 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:14.586127 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-2563b068-f978-45db-b018-6a5c3c1f2333 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:14.588983 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1212/2432] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:45:14 2022] DELETE /identity/v3/projects/cdc69db806d24d1f8eb18aadaaafabcb => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 07:45:14.596426 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db026beb-5fa8-496e-9aaa-5ffa29164967 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:14.596426 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db026beb-5fa8-496e-9aaa-5ffa29164967 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:14.596426 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db026beb-5fa8-496e-9aaa-5ffa29164967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:14.608233 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:45:14.627948 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-db026beb-5fa8-496e-9aaa-5ffa29164967 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:45:14.630404 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-db026beb-5fa8-496e-9aaa-5ffa29164967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:45:14.742711 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1221/2433] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:45:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:45:14.751534 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e359199-dfbb-4618-993b-5c9858443ddf None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:14.752170 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e359199-dfbb-4618-993b-5c9858443ddf None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:14.752523 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e359199-dfbb-4618-993b-5c9858443ddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:14.776420 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-3e359199-dfbb-4618-993b-5c9858443ddf None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:45:14.778394 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-3e359199-dfbb-4618-993b-5c9858443ddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:45:14.788043 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1213/2434] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:45:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:45:14.795379 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db026beb-5fa8-496e-9aaa-5ffa29164967 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:14.815443 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db026beb-5fa8-496e-9aaa-5ffa29164967 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:14.821353 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d97ff45-3230-4bc3-aab7-5efd200bb051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:14.822308 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d97ff45-3230-4bc3-aab7-5efd200bb051 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:14.826050 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d97ff45-3230-4bc3-aab7-5efd200bb051 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:14.826050 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d97ff45-3230-4bc3-aab7-5efd200bb051 admin admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:14.826050 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d97ff45-3230-4bc3-aab7-5efd200bb051 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:45:14.826050 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d97ff45-3230-4bc3-aab7-5efd200bb051 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:14.826050 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d97ff45-3230-4bc3-aab7-5efd200bb051 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:14.840157 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1222/2435] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 15 07:45:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:45:14.844760 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e359199-dfbb-4618-993b-5c9858443ddf None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:14.854470 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e359199-dfbb-4618-993b-5c9858443ddf None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:14.860431 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c14914-2925-4d44-b140-a0a870fe9ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:14.860431 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c14914-2925-4d44-b140-a0a870fe9ed8 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:14.860431 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c14914-2925-4d44-b140-a0a870fe9ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:14.860431 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c14914-2925-4d44-b140-a0a870fe9ed8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:14.860977 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c14914-2925-4d44-b140-a0a870fe9ed8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:14.862893 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c14914-2925-4d44-b140-a0a870fe9ed8 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:14.955669 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1214/2436] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:45:14 2022] POST /identity/v3/projects => generated 387 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:45:14.961206 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d97ff45-3230-4bc3-aab7-5efd200bb051 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:14.972279 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d97ff45-3230-4bc3-aab7-5efd200bb051 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:15.099633 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e73c219-69de-4511-8102-555b8531be39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:15.100256 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e73c219-69de-4511-8102-555b8531be39 admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:15.100529 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e73c219-69de-4511-8102-555b8531be39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:15.100830 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e73c219-69de-4511-8102-555b8531be39 admin admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:15.101463 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e73c219-69de-4511-8102-555b8531be39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:15.103431 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e73c219-69de-4511-8102-555b8531be39 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:15.206528 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1223/2437] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:45:14 2022] POST /identity/v3/users => generated 294 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:45:15.210779 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c14914-2925-4d44-b140-a0a870fe9ed8 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:15.225312 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c14914-2925-4d44-b140-a0a870fe9ed8 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:15.231878 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7f2731-4fb6-4ab8-bca4-4d122857c56f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:15.232496 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7f2731-4fb6-4ab8-bca4-4d122857c56f admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:15.232780 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7f2731-4fb6-4ab8-bca4-4d122857c56f admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:15.233056 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7f2731-4fb6-4ab8-bca4-4d122857c56f admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:15.234063 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7f2731-4fb6-4ab8-bca4-4d122857c56f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:15.235356 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7f2731-4fb6-4ab8-bca4-4d122857c56f admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:15.246652 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1215/2438] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:45:15 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:45:15.251089 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e73c219-69de-4511-8102-555b8531be39 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:15.261699 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e73c219-69de-4511-8102-555b8531be39 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:15.266242 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e679134d-7954-4e75-bbcb-fc74b97a10e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:15.266840 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e679134d-7954-4e75-bbcb-fc74b97a10e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:15.267123 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e679134d-7954-4e75-bbcb-fc74b97a10e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:15.270726 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e679134d-7954-4e75-bbcb-fc74b97a10e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:15.270726 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e679134d-7954-4e75-bbcb-fc74b97a10e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:15.270726 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e679134d-7954-4e75-bbcb-fc74b97a10e5 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:15.281116 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1224/2439] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:45:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:45:15.285316 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7f2731-4fb6-4ab8-bca4-4d122857c56f admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:15.296231 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7f2731-4fb6-4ab8-bca4-4d122857c56f admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:15.301506 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7815953e-87fd-4a7d-86b7-973da4281473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:15.301506 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7815953e-87fd-4a7d-86b7-973da4281473 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:15.301816 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7815953e-87fd-4a7d-86b7-973da4281473 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:15.301908 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7815953e-87fd-4a7d-86b7-973da4281473 admin admin] PATH_INFO: `/v3/projects/9be93644738a452ab16e847a56089b99/users/ed86fe3cc4c2472e84650b6fd2820127/roles/053e12af84e943e699460a9d76c2072b` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:15.310354 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:45:15.316628 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7815953e-87fd-4a7d-86b7-973da4281473 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9be93644738a452ab16e847a56089b99, user_id=ed86fe3cc4c2472e84650b6fd2820127, role_id=053e12af84e943e699460a9d76c2072b)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:15.319531 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7815953e-87fd-4a7d-86b7-973da4281473 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:15.408572 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1216/2440] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:45:15 2022] PUT /identity/v3/projects/9be93644738a452ab16e847a56089b99/users/ed86fe3cc4c2472e84650b6fd2820127/roles/053e12af84e943e699460a9d76c2072b => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:45:15.419460 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127776cb-f768-452b-9fea-ba198bb0375a None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:15.422060 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127776cb-f768-452b-9fea-ba198bb0375a None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:15.422060 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127776cb-f768-452b-9fea-ba198bb0375a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:15.467052 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-127776cb-f768-452b-9fea-ba198bb0375a None None] MFA Rules not processed for user `ed86fe3cc4c2472e84650b6fd2820127`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:45:15.468277 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-127776cb-f768-452b-9fea-ba198bb0375a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:45:15.552135 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:45:15.556488 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1225/2441] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:45:15 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:45:15.560965 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7815953e-87fd-4a7d-86b7-973da4281473 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:15.690542 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7815953e-87fd-4a7d-86b7-973da4281473 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:15.696851 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ce2d69-d268-41b8-a7c5-7742629b7eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:15.697955 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ce2d69-d268-41b8-a7c5-7742629b7eda admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:15.698249 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ce2d69-d268-41b8-a7c5-7742629b7eda admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:15.698524 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ce2d69-d268-41b8-a7c5-7742629b7eda admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:15.699181 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ce2d69-d268-41b8-a7c5-7742629b7eda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:15.701281 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ce2d69-d268-41b8-a7c5-7742629b7eda admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:15.813659 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1217/2442] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:45:15 2022] POST /identity/v3/projects => generated 385 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:45:15.817757 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127776cb-f768-452b-9fea-ba198bb0375a None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:15.831838 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127776cb-f768-452b-9fea-ba198bb0375a None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:15.891836 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6727f971-6965-4e30-bf03-8485878803ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:15.892452 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6727f971-6965-4e30-bf03-8485878803ce admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:15.893311 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6727f971-6965-4e30-bf03-8485878803ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:15.893647 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6727f971-6965-4e30-bf03-8485878803ce admin admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:15.897720 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6727f971-6965-4e30-bf03-8485878803ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:15.897720 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6727f971-6965-4e30-bf03-8485878803ce admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:15.957265 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1226/2443] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:45:15 2022] POST /identity/v3/users => generated 299 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:45:15.959297 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ce2d69-d268-41b8-a7c5-7742629b7eda admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:15.972457 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ce2d69-d268-41b8-a7c5-7742629b7eda admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:15.978017 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d908f6c3-298e-4859-900a-a49c02c38ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:15.978714 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d908f6c3-298e-4859-900a-a49c02c38ae3 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:15.979054 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d908f6c3-298e-4859-900a-a49c02c38ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:15.979506 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d908f6c3-298e-4859-900a-a49c02c38ae3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:15.980166 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d908f6c3-298e-4859-900a-a49c02c38ae3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:15.981771 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d908f6c3-298e-4859-900a-a49c02c38ae3 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:15.995007 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1218/2444] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:45:15 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:45:15.999150 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6727f971-6965-4e30-bf03-8485878803ce admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:16.010149 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6727f971-6965-4e30-bf03-8485878803ce admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:16.014835 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eac0316-d3b3-47ea-826b-5583f081f933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:16.015488 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eac0316-d3b3-47ea-826b-5583f081f933 admin admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:16.015768 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eac0316-d3b3-47ea-826b-5583f081f933 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:16.016063 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eac0316-d3b3-47ea-826b-5583f081f933 admin admin] PATH_INFO: `/v3/projects/a87ad6828f884a5292921863521c0f90/users/0ab6a735573641c39818712c9ccde9c5/roles/6bbb643e0d8d4bc0a9b27c26edf59216` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:16.031571 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eac0316-d3b3-47ea-826b-5583f081f933 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a87ad6828f884a5292921863521c0f90, user_id=0ab6a735573641c39818712c9ccde9c5, role_id=6bbb643e0d8d4bc0a9b27c26edf59216)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:16.034277 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eac0316-d3b3-47ea-826b-5583f081f933 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:16.142712 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1227/2445] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:45:15 2022] PUT /identity/v3/projects/a87ad6828f884a5292921863521c0f90/users/0ab6a735573641c39818712c9ccde9c5/roles/6bbb643e0d8d4bc0a9b27c26edf59216 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 07:45:16.147653 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d908f6c3-298e-4859-900a-a49c02c38ae3 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:16.157609 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d908f6c3-298e-4859-900a-a49c02c38ae3 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:16.217761 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa11aed5-aa92-4b20-83c9-6025f1d6c4d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:16.218431 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa11aed5-aa92-4b20-83c9-6025f1d6c4d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:16.218717 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa11aed5-aa92-4b20-83c9-6025f1d6c4d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:16.218995 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa11aed5-aa92-4b20-83c9-6025f1d6c4d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:16.219640 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa11aed5-aa92-4b20-83c9-6025f1d6c4d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:16.221230 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa11aed5-aa92-4b20-83c9-6025f1d6c4d4 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:16.232128 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1219/2446] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:45:16 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 07:45:16.236298 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eac0316-d3b3-47ea-826b-5583f081f933 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:16.247265 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eac0316-d3b3-47ea-826b-5583f081f933 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:16.253445 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9b6d36-3ed9-4e31-a34f-a419b9054852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:16.254201 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b6d36-3ed9-4e31-a34f-a419b9054852 admin admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:16.254480 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b6d36-3ed9-4e31-a34f-a419b9054852 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:16.254758 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b6d36-3ed9-4e31-a34f-a419b9054852 admin admin] PATH_INFO: `/v3/projects/a87ad6828f884a5292921863521c0f90/users/0ab6a735573641c39818712c9ccde9c5/roles/6bbb643e0d8d4bc0a9b27c26edf59216` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:16.258724 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9b6d36-3ed9-4e31-a34f-a419b9054852 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a87ad6828f884a5292921863521c0f90, user_id=0ab6a735573641c39818712c9ccde9c5, role_id=6bbb643e0d8d4bc0a9b27c26edf59216)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:16.261064 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9b6d36-3ed9-4e31-a34f-a419b9054852 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:16.286122 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1228/2447] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:45:16 2022] PUT /identity/v3/projects/a87ad6828f884a5292921863521c0f90/users/0ab6a735573641c39818712c9ccde9c5/roles/6bbb643e0d8d4bc0a9b27c26edf59216 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:45:16.287420 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d663fbe2-db48-47a4-a6c2-696aa1bfcbdd None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:16.287420 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d663fbe2-db48-47a4-a6c2-696aa1bfcbdd None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:16.287420 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d663fbe2-db48-47a4-a6c2-696aa1bfcbdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:16.330341 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-d663fbe2-db48-47a4-a6c2-696aa1bfcbdd None None] MFA Rules not processed for user `0ab6a735573641c39818712c9ccde9c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:45:16.331561 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-d663fbe2-db48-47a4-a6c2-696aa1bfcbdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:45:16.381089 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:45:16.407988 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1220/2448] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:45:16 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:45:16.425952 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9b6d36-3ed9-4e31-a34f-a419b9054852 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:16.430156 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-7c9b6d36-3ed9-4e31-a34f-a419b9054852 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:45:16.496072 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9b6d36-3ed9-4e31-a34f-a419b9054852 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:16.500135 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ff8fed-3a60-4dc1-82b8-2c56d31ff7ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecf6642b7b84d3c9756bb41ab3349d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:16.500738 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ff8fed-3a60-4dc1-82b8-2c56d31ff7ef service cinder] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:16.501033 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ff8fed-3a60-4dc1-82b8-2c56d31ff7ef service cinder] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:16.501286 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ff8fed-3a60-4dc1-82b8-2c56d31ff7ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:16.513838 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ff8fed-3a60-4dc1-82b8-2c56d31ff7ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:16.515402 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ff8fed-3a60-4dc1-82b8-2c56d31ff7ef service cinder] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:16.530423 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1229/2449] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:45:16 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:45:16.603413 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d663fbe2-db48-47a4-a6c2-696aa1bfcbdd None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:16.670970 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d663fbe2-db48-47a4-a6c2-696aa1bfcbdd None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:16.680233 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d45f579-48dd-45f7-8d94-4779f5015bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:16.681443 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d45f579-48dd-45f7-8d94-4779f5015bd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:16.681971 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d45f579-48dd-45f7-8d94-4779f5015bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:16.682453 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d45f579-48dd-45f7-8d94-4779f5015bd6 admin admin] PATH_INFO: `/v3/users/ed86fe3cc4c2472e84650b6fd2820127` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:16.683924 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d45f579-48dd-45f7-8d94-4779f5015bd6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed86fe3cc4c2472e84650b6fd2820127)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:16.686432 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d45f579-48dd-45f7-8d94-4779f5015bd6 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:16.846973 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ff8fed-3a60-4dc1-82b8-2c56d31ff7ef service cinder] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:16.849447 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-76ff8fed-3a60-4dc1-82b8-2c56d31ff7ef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:45:16.896378 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-6d45f579-48dd-45f7-8d94-4779f5015bd6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed86fe3cc4c2472e84650b6fd2820127'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:16.897560 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-6d45f579-48dd-45f7-8d94-4779f5015bd6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed86fe3cc4c2472e84650b6fd2820127'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:16.953324 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-6d45f579-48dd-45f7-8d94-4779f5015bd6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed86fe3cc4c2472e84650b6fd2820127'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:16.954782 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-6d45f579-48dd-45f7-8d94-4779f5015bd6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed86fe3cc4c2472e84650b6fd2820127'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:16.957429 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ff8fed-3a60-4dc1-82b8-2c56d31ff7ef service cinder] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:16.971121 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4178ee7-7161-47ec-83bf-e705b8f46708 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:16.971762 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4178ee7-7161-47ec-83bf-e705b8f46708 service neutron] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:16.972046 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4178ee7-7161-47ec-83bf-e705b8f46708 service neutron] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:16.972326 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4178ee7-7161-47ec-83bf-e705b8f46708 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:16.972491 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-6d45f579-48dd-45f7-8d94-4779f5015bd6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed86fe3cc4c2472e84650b6fd2820127'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:16.975808 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-d4178ee7-7161-47ec-83bf-e705b8f46708 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:45:16.993893 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4178ee7-7161-47ec-83bf-e705b8f46708 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:16.996381 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4178ee7-7161-47ec-83bf-e705b8f46708 service neutron] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:17.000222 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:45:17.006454 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1221/2450] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:45:16 2022] DELETE /identity/v3/users/ed86fe3cc4c2472e84650b6fd2820127 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:45:17.014145 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1230/2451] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:45:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 07:45:17.028627 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007f9781-3de0-42ad-9bb7-61c5621a488c None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:17.028919 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007f9781-3de0-42ad-9bb7-61c5621a488c None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:17.029199 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007f9781-3de0-42ad-9bb7-61c5621a488c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:17.064944 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-007f9781-3de0-42ad-9bb7-61c5621a488c None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:45:17.066153 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-007f9781-3de0-42ad-9bb7-61c5621a488c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:45:17.135488 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1222/2452] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:45:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:45:17.145291 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4178ee7-7161-47ec-83bf-e705b8f46708 service neutron] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:17.147376 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-d4178ee7-7161-47ec-83bf-e705b8f46708 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:45:17.157915 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4178ee7-7161-47ec-83bf-e705b8f46708 service neutron] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:17.213301 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350efbbf-9589-4b0b-8ce3-ce23b487a82e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:17.213949 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350efbbf-9589-4b0b-8ce3-ce23b487a82e service neutron] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:17.214255 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350efbbf-9589-4b0b-8ce3-ce23b487a82e service neutron] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:17.214526 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350efbbf-9589-4b0b-8ce3-ce23b487a82e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:17.225213 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350efbbf-9589-4b0b-8ce3-ce23b487a82e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:17.226835 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350efbbf-9589-4b0b-8ce3-ce23b487a82e service neutron] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:17.236403 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1231/2453] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:45:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:45:17.789556 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007f9781-3de0-42ad-9bb7-61c5621a488c None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:17.791571 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-007f9781-3de0-42ad-9bb7-61c5621a488c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:45:17.798800 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:45:17.805920 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007f9781-3de0-42ad-9bb7-61c5621a488c None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:17.811807 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a13e75e-a497-4b79-a080-96d4a329c7d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:17.812499 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a13e75e-a497-4b79-a080-96d4a329c7d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:17.812785 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a13e75e-a497-4b79-a080-96d4a329c7d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:17.816023 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a13e75e-a497-4b79-a080-96d4a329c7d3 admin admin] PATH_INFO: `/v3/projects/9be93644738a452ab16e847a56089b99` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:17.816023 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a13e75e-a497-4b79-a080-96d4a329c7d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9be93644738a452ab16e847a56089b99)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:17.816253 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a13e75e-a497-4b79-a080-96d4a329c7d3 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:17.936788 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-7a13e75e-a497-4b79-a080-96d4a329c7d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9be93644738a452ab16e847a56089b99', 'request_id': 'req-7a13e75e-a497-4b79-a080-96d4a329c7d3'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:17.946599 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-7a13e75e-a497-4b79-a080-96d4a329c7d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9be93644738a452ab16e847a56089b99', 'request_id': 'req-7a13e75e-a497-4b79-a080-96d4a329c7d3'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:18.047475 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-7a13e75e-a497-4b79-a080-96d4a329c7d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9be93644738a452ab16e847a56089b99', 'request_id': 'req-7a13e75e-a497-4b79-a080-96d4a329c7d3'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:18.061517 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-7a13e75e-a497-4b79-a080-96d4a329c7d3 admin admin] The token cache is being invalidate because project 9be93644738a452ab16e847a56089b99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:45:18.061855 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-7a13e75e-a497-4b79-a080-96d4a329c7d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:18.062706 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-7a13e75e-a497-4b79-a080-96d4a329c7d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:18.064578 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1223/2454] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:45:17 2022] DELETE /identity/v3/projects/9be93644738a452ab16e847a56089b99 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:45:18.069167 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350efbbf-9589-4b0b-8ce3-ce23b487a82e service neutron] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:18.081428 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350efbbf-9589-4b0b-8ce3-ce23b487a82e service neutron] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:18.143710 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de9d340-c0ba-4fad-bf9a-bf771ae6b811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:18.144353 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de9d340-c0ba-4fad-bf9a-bf771ae6b811 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:18.144745 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de9d340-c0ba-4fad-bf9a-bf771ae6b811 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:18.145025 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de9d340-c0ba-4fad-bf9a-bf771ae6b811 admin admin] PATH_INFO: `/v3/users/0ab6a735573641c39818712c9ccde9c5` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:18.146594 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de9d340-c0ba-4fad-bf9a-bf771ae6b811 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ab6a735573641c39818712c9ccde9c5)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:18.148527 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de9d340-c0ba-4fad-bf9a-bf771ae6b811 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:18.316604 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-6de9d340-c0ba-4fad-bf9a-bf771ae6b811 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ab6a735573641c39818712c9ccde9c5'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:18.387241 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-6de9d340-c0ba-4fad-bf9a-bf771ae6b811 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ab6a735573641c39818712c9ccde9c5'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:18.387930 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-6de9d340-c0ba-4fad-bf9a-bf771ae6b811 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ab6a735573641c39818712c9ccde9c5'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:18.388582 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-6de9d340-c0ba-4fad-bf9a-bf771ae6b811 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ab6a735573641c39818712c9ccde9c5'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:18.396640 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-6de9d340-c0ba-4fad-bf9a-bf771ae6b811 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ab6a735573641c39818712c9ccde9c5'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:18.415294 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1232/2455] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:45:18 2022] DELETE /identity/v3/users/0ab6a735573641c39818712c9ccde9c5 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:45:18.863105 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a13e75e-a497-4b79-a080-96d4a329c7d3 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:18.866562 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-7a13e75e-a497-4b79-a080-96d4a329c7d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:45:18.878917 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a13e75e-a497-4b79-a080-96d4a329c7d3 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:18.955012 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26fb431-bada-4911-a256-2278760955a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:18.955658 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26fb431-bada-4911-a256-2278760955a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:18.955939 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26fb431-bada-4911-a256-2278760955a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:18.956214 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26fb431-bada-4911-a256-2278760955a8 admin admin] PATH_INFO: `/v3/projects/a87ad6828f884a5292921863521c0f90` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:18.957331 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26fb431-bada-4911-a256-2278760955a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a87ad6828f884a5292921863521c0f90)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:18.959825 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26fb431-bada-4911-a256-2278760955a8 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:19.061511 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-d26fb431-bada-4911-a256-2278760955a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a87ad6828f884a5292921863521c0f90', 'request_id': 'req-d26fb431-bada-4911-a256-2278760955a8'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:19.067986 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-d26fb431-bada-4911-a256-2278760955a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a87ad6828f884a5292921863521c0f90', 'request_id': 'req-d26fb431-bada-4911-a256-2278760955a8'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:19.134736 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-d26fb431-bada-4911-a256-2278760955a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a87ad6828f884a5292921863521c0f90', 'request_id': 'req-d26fb431-bada-4911-a256-2278760955a8'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:19.146557 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-d26fb431-bada-4911-a256-2278760955a8 admin admin] The token cache is being invalidate because project a87ad6828f884a5292921863521c0f90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:45:19.146875 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-d26fb431-bada-4911-a256-2278760955a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:19.147527 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-d26fb431-bada-4911-a256-2278760955a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:19.149012 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1224/2456] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:45:18 2022] DELETE /identity/v3/projects/a87ad6828f884a5292921863521c0f90 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:45:19.153898 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40f07a6-7a2f-46aa-8262-06bacb9d7dbd None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:19.154372 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40f07a6-7a2f-46aa-8262-06bacb9d7dbd None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:19.154518 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40f07a6-7a2f-46aa-8262-06bacb9d7dbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:19.174724 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-e40f07a6-7a2f-46aa-8262-06bacb9d7dbd None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:45:19.175920 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-e40f07a6-7a2f-46aa-8262-06bacb9d7dbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:45:19.189749 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:45:19.244195 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1233/2457] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:45:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:45:19.253265 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb791d8-8b7a-4365-b62b-aad1f415ff2f None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:19.253762 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb791d8-8b7a-4365-b62b-aad1f415ff2f None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:19.254138 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb791d8-8b7a-4365-b62b-aad1f415ff2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:19.280612 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-cbb791d8-8b7a-4365-b62b-aad1f415ff2f None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:45:19.281894 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-cbb791d8-8b7a-4365-b62b-aad1f415ff2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:45:19.291145 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1225/2458] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:45:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 07:45:19.295492 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40f07a6-7a2f-46aa-8262-06bacb9d7dbd None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:19.305532 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40f07a6-7a2f-46aa-8262-06bacb9d7dbd None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:19.310240 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73620368-f8b4-4735-834b-69fc2b52fe2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:19.310856 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73620368-f8b4-4735-834b-69fc2b52fe2f admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:19.311138 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73620368-f8b4-4735-834b-69fc2b52fe2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:19.311413 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73620368-f8b4-4735-834b-69fc2b52fe2f admin admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:19.312066 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73620368-f8b4-4735-834b-69fc2b52fe2f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:45:19.316036 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73620368-f8b4-4735-834b-69fc2b52fe2f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:19.316036 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73620368-f8b4-4735-834b-69fc2b52fe2f admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:19.327052 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1234/2459] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 15 07:45:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:45:19.331235 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb791d8-8b7a-4365-b62b-aad1f415ff2f None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:19.341359 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb791d8-8b7a-4365-b62b-aad1f415ff2f None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:19.346273 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b97e37a-f546-4c8d-87a2-75bec1ffe0a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:19.346959 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b97e37a-f546-4c8d-87a2-75bec1ffe0a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:19.347312 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b97e37a-f546-4c8d-87a2-75bec1ffe0a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:19.347680 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b97e37a-f546-4c8d-87a2-75bec1ffe0a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:19.348475 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b97e37a-f546-4c8d-87a2-75bec1ffe0a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:19.350711 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b97e37a-f546-4c8d-87a2-75bec1ffe0a8 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:19.356421 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:45:19.463541 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1226/2460] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:45:19 2022] POST /identity/v3/projects => generated 379 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:45:19.468293 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73620368-f8b4-4735-834b-69fc2b52fe2f admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:19.479901 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73620368-f8b4-4735-834b-69fc2b52fe2f admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:19.542944 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c9dfa3-b266-4665-83cb-d933a66ee7ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:19.543565 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c9dfa3-b266-4665-83cb-d933a66ee7ed admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:19.543848 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c9dfa3-b266-4665-83cb-d933a66ee7ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:19.544124 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c9dfa3-b266-4665-83cb-d933a66ee7ed admin admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:19.544896 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c9dfa3-b266-4665-83cb-d933a66ee7ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:19.546876 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c9dfa3-b266-4665-83cb-d933a66ee7ed admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:19.622256 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1235/2461] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:45:19 2022] POST /identity/v3/users => generated 290 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:45:19.626376 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b97e37a-f546-4c8d-87a2-75bec1ffe0a8 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:19.637409 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b97e37a-f546-4c8d-87a2-75bec1ffe0a8 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:19.641982 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c9cf1d-a1b6-4b68-9e25-1b938c66f671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:19.642713 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c9cf1d-a1b6-4b68-9e25-1b938c66f671 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:19.642996 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c9cf1d-a1b6-4b68-9e25-1b938c66f671 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:19.643273 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c9cf1d-a1b6-4b68-9e25-1b938c66f671 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:19.643910 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c9cf1d-a1b6-4b68-9e25-1b938c66f671 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:19.645490 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c9cf1d-a1b6-4b68-9e25-1b938c66f671 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:19.655702 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1227/2462] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:45:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:45:19.659902 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c9dfa3-b266-4665-83cb-d933a66ee7ed admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:19.670123 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c9dfa3-b266-4665-83cb-d933a66ee7ed admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:19.674784 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3566fd31-6858-4b1d-938d-4be238ce9951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:19.675389 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3566fd31-6858-4b1d-938d-4be238ce9951 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:19.675672 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3566fd31-6858-4b1d-938d-4be238ce9951 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:19.675952 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3566fd31-6858-4b1d-938d-4be238ce9951 admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:19.677466 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3566fd31-6858-4b1d-938d-4be238ce9951 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:19.679116 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3566fd31-6858-4b1d-938d-4be238ce9951 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:19.690971 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1236/2463] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:45:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:45:19.697297 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c9cf1d-a1b6-4b68-9e25-1b938c66f671 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:19.707496 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c9cf1d-a1b6-4b68-9e25-1b938c66f671 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:19.713115 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee46841-4f55-4c6c-9e14-6e15953d26f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:19.713803 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee46841-4f55-4c6c-9e14-6e15953d26f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:19.714089 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee46841-4f55-4c6c-9e14-6e15953d26f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:19.714470 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee46841-4f55-4c6c-9e14-6e15953d26f4 admin admin] PATH_INFO: `/v3/projects/d8c88be168a247f3a1d7a268bf3d4093/users/76a80b4388334688ac6d781cad804028/roles/053e12af84e943e699460a9d76c2072b` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:19.731704 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee46841-4f55-4c6c-9e14-6e15953d26f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8c88be168a247f3a1d7a268bf3d4093, user_id=76a80b4388334688ac6d781cad804028, role_id=053e12af84e943e699460a9d76c2072b)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:19.734484 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee46841-4f55-4c6c-9e14-6e15953d26f4 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:19.798586 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1228/2464] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:45:19 2022] PUT /identity/v3/projects/d8c88be168a247f3a1d7a268bf3d4093/users/76a80b4388334688ac6d781cad804028/roles/053e12af84e943e699460a9d76c2072b => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 07:45:19.808683 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c430fe4b-4bbf-43da-97ee-0b57e3d1b52c None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:19.808972 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c430fe4b-4bbf-43da-97ee-0b57e3d1b52c None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:19.809244 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c430fe4b-4bbf-43da-97ee-0b57e3d1b52c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:19.847397 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-c430fe4b-4bbf-43da-97ee-0b57e3d1b52c None None] MFA Rules not processed for user `76a80b4388334688ac6d781cad804028`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:45:19.850237 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-c430fe4b-4bbf-43da-97ee-0b57e3d1b52c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:45:19.949513 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1237/2465] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:45:19 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 07:45:19.954109 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee46841-4f55-4c6c-9e14-6e15953d26f4 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:20.061224 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee46841-4f55-4c6c-9e14-6e15953d26f4 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:20.068455 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f146cb-2eaa-4fe4-a5b9-8b4d917e15fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:20.068455 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f146cb-2eaa-4fe4-a5b9-8b4d917e15fe admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:20.068455 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f146cb-2eaa-4fe4-a5b9-8b4d917e15fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:20.068455 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f146cb-2eaa-4fe4-a5b9-8b4d917e15fe admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:20.071358 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f146cb-2eaa-4fe4-a5b9-8b4d917e15fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:20.071358 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f146cb-2eaa-4fe4-a5b9-8b4d917e15fe admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:20.178902 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1229/2466] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:45:19 2022] POST /identity/v3/projects => generated 379 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:45:20.183101 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c430fe4b-4bbf-43da-97ee-0b57e3d1b52c None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:20.195957 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c430fe4b-4bbf-43da-97ee-0b57e3d1b52c None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:20.209135 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:45:20.268598 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7077b721-e80f-4019-a998-7f009b5c9909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:20.269212 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7077b721-e80f-4019-a998-7f009b5c9909 admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:20.269492 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7077b721-e80f-4019-a998-7f009b5c9909 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:20.269797 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7077b721-e80f-4019-a998-7f009b5c9909 admin admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:20.270764 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7077b721-e80f-4019-a998-7f009b5c9909 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:20.272676 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7077b721-e80f-4019-a998-7f009b5c9909 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:20.358584 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1238/2467] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:45:20 2022] POST /identity/v3/users => generated 296 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 07:45:20.362816 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f146cb-2eaa-4fe4-a5b9-8b4d917e15fe admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:20.374285 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f146cb-2eaa-4fe4-a5b9-8b4d917e15fe admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:20.379096 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f66bff9-82e2-45ea-b5f4-e493d87ddc4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:20.380607 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f66bff9-82e2-45ea-b5f4-e493d87ddc4b admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:20.385512 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f66bff9-82e2-45ea-b5f4-e493d87ddc4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:20.385512 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f66bff9-82e2-45ea-b5f4-e493d87ddc4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:20.385512 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f66bff9-82e2-45ea-b5f4-e493d87ddc4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:20.385512 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f66bff9-82e2-45ea-b5f4-e493d87ddc4b admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:20.395651 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1230/2468] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:45:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 07:45:20.399652 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7077b721-e80f-4019-a998-7f009b5c9909 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:20.410065 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7077b721-e80f-4019-a998-7f009b5c9909 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:20.414982 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cdb647-79f0-44cd-a105-4be4269fca05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:20.415845 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cdb647-79f0-44cd-a105-4be4269fca05 admin admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:20.416132 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cdb647-79f0-44cd-a105-4be4269fca05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:20.416411 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cdb647-79f0-44cd-a105-4be4269fca05 admin admin] PATH_INFO: `/v3/projects/063d04b2bd3343ddbb30bdeeb41c3b16/users/3e37e899de464e9faf21d1d3e487a438/roles/6bbb643e0d8d4bc0a9b27c26edf59216` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:20.434733 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cdb647-79f0-44cd-a105-4be4269fca05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=063d04b2bd3343ddbb30bdeeb41c3b16, user_id=3e37e899de464e9faf21d1d3e487a438, role_id=6bbb643e0d8d4bc0a9b27c26edf59216)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:20.438065 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cdb647-79f0-44cd-a105-4be4269fca05 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:20.522260 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1239/2469] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:45:20 2022] PUT /identity/v3/projects/063d04b2bd3343ddbb30bdeeb41c3b16/users/3e37e899de464e9faf21d1d3e487a438/roles/6bbb643e0d8d4bc0a9b27c26edf59216 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:45:20.526628 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f66bff9-82e2-45ea-b5f4-e493d87ddc4b admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:20.540036 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f66bff9-82e2-45ea-b5f4-e493d87ddc4b admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:20.623820 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0baad3a-f36e-4210-b411-84c4dd2a17cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:20.624524 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0baad3a-f36e-4210-b411-84c4dd2a17cf admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:20.624787 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0baad3a-f36e-4210-b411-84c4dd2a17cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:20.625205 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0baad3a-f36e-4210-b411-84c4dd2a17cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:20.626048 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0baad3a-f36e-4210-b411-84c4dd2a17cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:20.627695 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0baad3a-f36e-4210-b411-84c4dd2a17cf admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:20.648008 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1231/2470] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:45:20 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 07:45:20.651880 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cdb647-79f0-44cd-a105-4be4269fca05 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:20.663300 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cdb647-79f0-44cd-a105-4be4269fca05 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:20.670199 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5991d68c-388c-4d16-a89c-7a289c81c1bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:20.670871 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5991d68c-388c-4d16-a89c-7a289c81c1bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:20.671153 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5991d68c-388c-4d16-a89c-7a289c81c1bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:20.671841 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5991d68c-388c-4d16-a89c-7a289c81c1bf admin admin] PATH_INFO: `/v3/projects/063d04b2bd3343ddbb30bdeeb41c3b16/users/3e37e899de464e9faf21d1d3e487a438/roles/6bbb643e0d8d4bc0a9b27c26edf59216` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:20.673388 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5991d68c-388c-4d16-a89c-7a289c81c1bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=063d04b2bd3343ddbb30bdeeb41c3b16, user_id=3e37e899de464e9faf21d1d3e487a438, role_id=6bbb643e0d8d4bc0a9b27c26edf59216)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:20.676051 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5991d68c-388c-4d16-a89c-7a289c81c1bf admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:20.686695 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1240/2471] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:45:20 2022] PUT /identity/v3/projects/063d04b2bd3343ddbb30bdeeb41c3b16/users/3e37e899de464e9faf21d1d3e487a438/roles/6bbb643e0d8d4bc0a9b27c26edf59216 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:45:20.699148 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c62446-19d4-46e7-a4a7-06488b82909c None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:20.699505 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c62446-19d4-46e7-a4a7-06488b82909c None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:20.699855 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c62446-19d4-46e7-a4a7-06488b82909c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:20.732461 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-06c62446-19d4-46e7-a4a7-06488b82909c None None] MFA Rules not processed for user `3e37e899de464e9faf21d1d3e487a438`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:45:20.733734 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-06c62446-19d4-46e7-a4a7-06488b82909c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:45:20.745627 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:45:20.820395 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1232/2472] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:45:20 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:45:20.837123 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5991d68c-388c-4d16-a89c-7a289c81c1bf admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:20.839279 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-5991d68c-388c-4d16-a89c-7a289c81c1bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:45:20.896907 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5991d68c-388c-4d16-a89c-7a289c81c1bf admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:20.900579 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66116ebb-a5b0-4356-9767-d46fc05355fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecf6642b7b84d3c9756bb41ab3349d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:20.901120 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66116ebb-a5b0-4356-9767-d46fc05355fb service cinder] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:20.901398 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66116ebb-a5b0-4356-9767-d46fc05355fb service cinder] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:20.901749 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66116ebb-a5b0-4356-9767-d46fc05355fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:20.911589 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66116ebb-a5b0-4356-9767-d46fc05355fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:20.912678 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66116ebb-a5b0-4356-9767-d46fc05355fb service cinder] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:20.922035 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1241/2473] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:45:20 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:45:21.441517 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c62446-19d4-46e7-a4a7-06488b82909c None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:21.499775 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66116ebb-a5b0-4356-9767-d46fc05355fb service cinder] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:21.501444 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-66116ebb-a5b0-4356-9767-d46fc05355fb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:45:21.522495 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c62446-19d4-46e7-a4a7-06488b82909c None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:21.527654 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6929fdf5-e554-4c12-93a4-0581d287e67c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:21.528504 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6929fdf5-e554-4c12-93a4-0581d287e67c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:21.528785 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6929fdf5-e554-4c12-93a4-0581d287e67c admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:21.529055 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6929fdf5-e554-4c12-93a4-0581d287e67c admin admin] PATH_INFO: `/v3/users/76a80b4388334688ac6d781cad804028` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:21.530282 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6929fdf5-e554-4c12-93a4-0581d287e67c admin admin] RBAC: Authorizing `identity:delete_user(user_id=76a80b4388334688ac6d781cad804028)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:21.532248 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6929fdf5-e554-4c12-93a4-0581d287e67c admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:21.562809 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66116ebb-a5b0-4356-9767-d46fc05355fb service cinder] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:21.567474 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9750015d-e78c-4fd1-a79e-9f848870c968 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:21.568388 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9750015d-e78c-4fd1-a79e-9f848870c968 service neutron] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:21.568668 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9750015d-e78c-4fd1-a79e-9f848870c968 service neutron] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:21.568941 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9750015d-e78c-4fd1-a79e-9f848870c968 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:21.570830 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-9750015d-e78c-4fd1-a79e-9f848870c968 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:45:21.584604 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9750015d-e78c-4fd1-a79e-9f848870c968 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:21.586596 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9750015d-e78c-4fd1-a79e-9f848870c968 service neutron] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:21.598073 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1242/2474] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:45:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 07:45:21.813193 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-6929fdf5-e554-4c12-93a4-0581d287e67c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76a80b4388334688ac6d781cad804028'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:21.813956 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-6929fdf5-e554-4c12-93a4-0581d287e67c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76a80b4388334688ac6d781cad804028'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:21.860384 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-6929fdf5-e554-4c12-93a4-0581d287e67c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76a80b4388334688ac6d781cad804028'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:21.861327 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-6929fdf5-e554-4c12-93a4-0581d287e67c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76a80b4388334688ac6d781cad804028'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:21.870496 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-6929fdf5-e554-4c12-93a4-0581d287e67c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76a80b4388334688ac6d781cad804028'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:21.890936 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1233/2475] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:45:21 2022] DELETE /identity/v3/users/76a80b4388334688ac6d781cad804028 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:45:21.895954 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c22484-e62c-4f72-a05c-16e7dae933dd None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:21.896609 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c22484-e62c-4f72-a05c-16e7dae933dd None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:21.896958 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c22484-e62c-4f72-a05c-16e7dae933dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:21.919119 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-08c22484-e62c-4f72-a05c-16e7dae933dd None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:45:21.920324 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-08c22484-e62c-4f72-a05c-16e7dae933dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:45:21.930745 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:45:21.986464 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1243/2476] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:45:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:45:21.996981 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6929fdf5-e554-4c12-93a4-0581d287e67c admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:21.998554 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-6929fdf5-e554-4c12-93a4-0581d287e67c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:45:22.030462 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:45:22.058591 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6929fdf5-e554-4c12-93a4-0581d287e67c admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:22.062258 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bcc968-ec40-4844-9db6-ac14497a818e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:22.065369 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bcc968-ec40-4844-9db6-ac14497a818e service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:22.065369 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bcc968-ec40-4844-9db6-ac14497a818e service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:22.065369 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bcc968-ec40-4844-9db6-ac14497a818e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:22.073827 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bcc968-ec40-4844-9db6-ac14497a818e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:22.075353 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bcc968-ec40-4844-9db6-ac14497a818e service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:22.086432 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1234/2477] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:45:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:45:22.605231 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c22484-e62c-4f72-a05c-16e7dae933dd None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:22.606688 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-08c22484-e62c-4f72-a05c-16e7dae933dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:45:22.633964 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c22484-e62c-4f72-a05c-16e7dae933dd None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:22.639089 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903a4aa6-b2a4-49c3-9351-bc5a138046d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:22.639745 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903a4aa6-b2a4-49c3-9351-bc5a138046d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:22.640021 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903a4aa6-b2a4-49c3-9351-bc5a138046d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:22.640294 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903a4aa6-b2a4-49c3-9351-bc5a138046d8 admin admin] PATH_INFO: `/v3/projects/d8c88be168a247f3a1d7a268bf3d4093` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:22.641428 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903a4aa6-b2a4-49c3-9351-bc5a138046d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8c88be168a247f3a1d7a268bf3d4093)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:22.648838 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903a4aa6-b2a4-49c3-9351-bc5a138046d8 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:22.746303 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-903a4aa6-b2a4-49c3-9351-bc5a138046d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8c88be168a247f3a1d7a268bf3d4093', 'request_id': 'req-903a4aa6-b2a4-49c3-9351-bc5a138046d8'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:22.753816 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-903a4aa6-b2a4-49c3-9351-bc5a138046d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8c88be168a247f3a1d7a268bf3d4093', 'request_id': 'req-903a4aa6-b2a4-49c3-9351-bc5a138046d8'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:22.797806 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-903a4aa6-b2a4-49c3-9351-bc5a138046d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8c88be168a247f3a1d7a268bf3d4093', 'request_id': 'req-903a4aa6-b2a4-49c3-9351-bc5a138046d8'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:22.810173 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-903a4aa6-b2a4-49c3-9351-bc5a138046d8 admin admin] The token cache is being invalidate because project d8c88be168a247f3a1d7a268bf3d4093 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:45:22.810549 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-903a4aa6-b2a4-49c3-9351-bc5a138046d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:22.811295 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-903a4aa6-b2a4-49c3-9351-bc5a138046d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:22.812845 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1244/2478] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:45:22 2022] DELETE /identity/v3/projects/d8c88be168a247f3a1d7a268bf3d4093 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 07:45:22.817518 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bcc968-ec40-4844-9db6-ac14497a818e service neutron] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:22.827660 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bcc968-ec40-4844-9db6-ac14497a818e service neutron] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:22.890675 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e0fe92-bf52-4d45-945f-e9e61d373d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:22.891252 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e0fe92-bf52-4d45-945f-e9e61d373d6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:22.891536 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e0fe92-bf52-4d45-945f-e9e61d373d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:22.891848 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e0fe92-bf52-4d45-945f-e9e61d373d6e admin admin] PATH_INFO: `/v3/users/3e37e899de464e9faf21d1d3e487a438` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:22.893083 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e0fe92-bf52-4d45-945f-e9e61d373d6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e37e899de464e9faf21d1d3e487a438)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:22.895715 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e0fe92-bf52-4d45-945f-e9e61d373d6e admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:23.078382 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-e3e0fe92-bf52-4d45-945f-e9e61d373d6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e37e899de464e9faf21d1d3e487a438'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:23.079134 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-e3e0fe92-bf52-4d45-945f-e9e61d373d6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e37e899de464e9faf21d1d3e487a438'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:23.146056 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-e3e0fe92-bf52-4d45-945f-e9e61d373d6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e37e899de464e9faf21d1d3e487a438'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:23.146637 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-e3e0fe92-bf52-4d45-945f-e9e61d373d6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e37e899de464e9faf21d1d3e487a438'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:23.156183 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-e3e0fe92-bf52-4d45-945f-e9e61d373d6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e37e899de464e9faf21d1d3e487a438'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:23.175005 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1235/2479] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:45:22 2022] DELETE /identity/v3/users/3e37e899de464e9faf21d1d3e487a438 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 07:45:23.631349 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903a4aa6-b2a4-49c3-9351-bc5a138046d8 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:23.633522 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-903a4aa6-b2a4-49c3-9351-bc5a138046d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:45:23.648486 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903a4aa6-b2a4-49c3-9351-bc5a138046d8 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:23.734360 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e063fd20-d1bf-44a4-9ac5-ed2448a0b308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:23.735093 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e063fd20-d1bf-44a4-9ac5-ed2448a0b308 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:23.735451 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e063fd20-d1bf-44a4-9ac5-ed2448a0b308 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:23.735806 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e063fd20-d1bf-44a4-9ac5-ed2448a0b308 admin admin] PATH_INFO: `/v3/projects/063d04b2bd3343ddbb30bdeeb41c3b16` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:23.737301 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e063fd20-d1bf-44a4-9ac5-ed2448a0b308 admin admin] RBAC: Authorizing `identity:delete_project(project_id=063d04b2bd3343ddbb30bdeeb41c3b16)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:23.739936 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e063fd20-d1bf-44a4-9ac5-ed2448a0b308 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:23.835270 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:45:23.842738 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-e063fd20-d1bf-44a4-9ac5-ed2448a0b308 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '063d04b2bd3343ddbb30bdeeb41c3b16', 'request_id': 'req-e063fd20-d1bf-44a4-9ac5-ed2448a0b308'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:23.849749 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-e063fd20-d1bf-44a4-9ac5-ed2448a0b308 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '063d04b2bd3343ddbb30bdeeb41c3b16', 'request_id': 'req-e063fd20-d1bf-44a4-9ac5-ed2448a0b308'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:23.923439 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-e063fd20-d1bf-44a4-9ac5-ed2448a0b308 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '063d04b2bd3343ddbb30bdeeb41c3b16', 'request_id': 'req-e063fd20-d1bf-44a4-9ac5-ed2448a0b308'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:23.954580 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-e063fd20-d1bf-44a4-9ac5-ed2448a0b308 admin admin] The token cache is being invalidate because project 063d04b2bd3343ddbb30bdeeb41c3b16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:45:23.954875 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-e063fd20-d1bf-44a4-9ac5-ed2448a0b308 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:23.955556 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-e063fd20-d1bf-44a4-9ac5-ed2448a0b308 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:23.957889 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1245/2480] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:45:23 2022] DELETE /identity/v3/projects/063d04b2bd3343ddbb30bdeeb41c3b16 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 07:45:23.962957 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070f4829-5521-4c36-940e-c4e9102177a6 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:23.963422 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070f4829-5521-4c36-940e-c4e9102177a6 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:23.963612 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070f4829-5521-4c36-940e-c4e9102177a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:23.983554 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-070f4829-5521-4c36-940e-c4e9102177a6 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:45:23.984887 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-070f4829-5521-4c36-940e-c4e9102177a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:45:24.030158 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:45:24.044689 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1236/2481] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:45:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 07:45:24.052812 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658ff043-0f3e-4419-95cc-2fe8845db649 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:24.053096 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658ff043-0f3e-4419-95cc-2fe8845db649 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:24.053373 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658ff043-0f3e-4419-95cc-2fe8845db649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:24.073840 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-658ff043-0f3e-4419-95cc-2fe8845db649 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:45:24.075007 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-658ff043-0f3e-4419-95cc-2fe8845db649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:45:24.083484 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1246/2482] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:45:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:45:24.087803 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070f4829-5521-4c36-940e-c4e9102177a6 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:24.097096 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070f4829-5521-4c36-940e-c4e9102177a6 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:24.102094 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3058eb-4328-4c4a-91d2-ef84deff322a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:24.102609 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3058eb-4328-4c4a-91d2-ef84deff322a admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:24.105128 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3058eb-4328-4c4a-91d2-ef84deff322a admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:24.105128 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3058eb-4328-4c4a-91d2-ef84deff322a admin admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:24.105128 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3058eb-4328-4c4a-91d2-ef84deff322a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:45:24.105128 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3058eb-4328-4c4a-91d2-ef84deff322a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:24.105670 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3058eb-4328-4c4a-91d2-ef84deff322a admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:24.120221 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1237/2483] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 15 07:45:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:45:24.124850 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658ff043-0f3e-4419-95cc-2fe8845db649 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:24.135466 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658ff043-0f3e-4419-95cc-2fe8845db649 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:24.140458 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b23ab64-2922-43b6-ae6c-5ca5bd1d02c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:24.141155 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b23ab64-2922-43b6-ae6c-5ca5bd1d02c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:24.141496 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b23ab64-2922-43b6-ae6c-5ca5bd1d02c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:24.141809 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b23ab64-2922-43b6-ae6c-5ca5bd1d02c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:24.145158 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b23ab64-2922-43b6-ae6c-5ca5bd1d02c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:24.145158 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b23ab64-2922-43b6-ae6c-5ca5bd1d02c4 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:24.251085 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1247/2484] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:45:24 2022] POST /identity/v3/projects => generated 373 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 07:45:24.252630 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3058eb-4328-4c4a-91d2-ef84deff322a admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:24.268867 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3058eb-4328-4c4a-91d2-ef84deff322a admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:24.342021 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d28421-1fd9-4d85-a34e-813290c815e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:24.343156 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d28421-1fd9-4d85-a34e-813290c815e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:24.343565 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d28421-1fd9-4d85-a34e-813290c815e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:24.343721 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d28421-1fd9-4d85-a34e-813290c815e8 admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:24.344329 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d28421-1fd9-4d85-a34e-813290c815e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:24.346257 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d28421-1fd9-4d85-a34e-813290c815e8 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:24.437719 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1238/2485] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:45:24 2022] POST /identity/v3/users => generated 287 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:45:24.442416 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b23ab64-2922-43b6-ae6c-5ca5bd1d02c4 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:24.452814 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b23ab64-2922-43b6-ae6c-5ca5bd1d02c4 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:24.458166 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ed21d4-5a94-4388-9a7d-759c7cc12744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:24.458827 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ed21d4-5a94-4388-9a7d-759c7cc12744 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:24.459113 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ed21d4-5a94-4388-9a7d-759c7cc12744 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:24.459391 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ed21d4-5a94-4388-9a7d-759c7cc12744 admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:24.460039 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ed21d4-5a94-4388-9a7d-759c7cc12744 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:24.461689 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ed21d4-5a94-4388-9a7d-759c7cc12744 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:24.472111 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1248/2486] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:45:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:45:24.476202 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d28421-1fd9-4d85-a34e-813290c815e8 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:24.486593 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d28421-1fd9-4d85-a34e-813290c815e8 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:24.491403 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4868107e-312e-4d66-9b3c-a2001f78cc0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:24.492061 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4868107e-312e-4d66-9b3c-a2001f78cc0b admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:24.492283 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4868107e-312e-4d66-9b3c-a2001f78cc0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:24.492607 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4868107e-312e-4d66-9b3c-a2001f78cc0b admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:24.493278 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4868107e-312e-4d66-9b3c-a2001f78cc0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:24.494898 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4868107e-312e-4d66-9b3c-a2001f78cc0b admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:24.506217 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1239/2487] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:45:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:45:24.510605 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ed21d4-5a94-4388-9a7d-759c7cc12744 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:24.522085 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ed21d4-5a94-4388-9a7d-759c7cc12744 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:24.526847 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd575046-b03a-4bcd-8890-c4abee9938ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:24.527601 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd575046-b03a-4bcd-8890-c4abee9938ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:24.527885 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd575046-b03a-4bcd-8890-c4abee9938ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:24.528165 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd575046-b03a-4bcd-8890-c4abee9938ef admin admin] PATH_INFO: `/v3/projects/6a599d6dd6a34afabf5a156ba7f3981e/users/28c5b285ac03442d856446e848ba895a/roles/053e12af84e943e699460a9d76c2072b` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:24.543577 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd575046-b03a-4bcd-8890-c4abee9938ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a599d6dd6a34afabf5a156ba7f3981e, user_id=28c5b285ac03442d856446e848ba895a, role_id=053e12af84e943e699460a9d76c2072b)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:24.546591 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd575046-b03a-4bcd-8890-c4abee9938ef admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:24.657118 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1249/2488] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:45:24 2022] PUT /identity/v3/projects/6a599d6dd6a34afabf5a156ba7f3981e/users/28c5b285ac03442d856446e848ba895a/roles/053e12af84e943e699460a9d76c2072b => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:45:24.667948 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c7502e-7ef5-43aa-a07a-fd0c743f97a6 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:24.668288 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c7502e-7ef5-43aa-a07a-fd0c743f97a6 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:24.668567 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c7502e-7ef5-43aa-a07a-fd0c743f97a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:24.701998 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-70c7502e-7ef5-43aa-a07a-fd0c743f97a6 None None] MFA Rules not processed for user `28c5b285ac03442d856446e848ba895a`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:45:24.703329 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-70c7502e-7ef5-43aa-a07a-fd0c743f97a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:45:24.814552 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1240/2489] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:45:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 07:45:24.824548 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd575046-b03a-4bcd-8890-c4abee9938ef admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:24.826338 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-bd575046-b03a-4bcd-8890-c4abee9938ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:45:24.918992 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd575046-b03a-4bcd-8890-c4abee9938ef admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:24.923846 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10a4ae0-a5f4-42f1-b90d-50672412d18b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37438b032a1f46d3bdf7121ad0cffbc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:24.924804 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10a4ae0-a5f4-42f1-b90d-50672412d18b service glance] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:24.925153 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10a4ae0-a5f4-42f1-b90d-50672412d18b service glance] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:24.927355 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10a4ae0-a5f4-42f1-b90d-50672412d18b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:24.940507 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10a4ae0-a5f4-42f1-b90d-50672412d18b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:24.943064 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10a4ae0-a5f4-42f1-b90d-50672412d18b service glance] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:24.954632 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1250/2490] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:45:24 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:45:25.446443 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c7502e-7ef5-43aa-a07a-fd0c743f97a6 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:25.447887 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-70c7502e-7ef5-43aa-a07a-fd0c743f97a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:45:25.498248 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:45:25.537465 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c7502e-7ef5-43aa-a07a-fd0c743f97a6 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:25.543427 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eaedd7-ec8f-4e20-adfd-77ce4d6a26c7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd87846b9a134248b28e0521b0a880c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:25.544053 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eaedd7-ec8f-4e20-adfd-77ce4d6a26c7 service placement] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:25.544780 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eaedd7-ec8f-4e20-adfd-77ce4d6a26c7 service placement] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:25.544780 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eaedd7-ec8f-4e20-adfd-77ce4d6a26c7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:25.546478 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-c1eaedd7-ec8f-4e20-adfd-77ce4d6a26c7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:45:25.561825 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eaedd7-ec8f-4e20-adfd-77ce4d6a26c7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:25.563559 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eaedd7-ec8f-4e20-adfd-77ce4d6a26c7 service placement] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:25.574935 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1241/2491] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:45:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:45:25.630145 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10a4ae0-a5f4-42f1-b90d-50672412d18b service glance] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:25.632212 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-b10a4ae0-a5f4-42f1-b90d-50672412d18b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:45:25.712606 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10a4ae0-a5f4-42f1-b90d-50672412d18b service glance] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:25.717349 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e87030-4ad3-40b4-be54-57a1a1977135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:25.718019 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e87030-4ad3-40b4-be54-57a1a1977135 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:25.718445 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e87030-4ad3-40b4-be54-57a1a1977135 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:25.718715 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e87030-4ad3-40b4-be54-57a1a1977135 admin admin] PATH_INFO: `/v3/users/5e0c8d7aa4db455a94631becf212866f` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:25.719891 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e87030-4ad3-40b4-be54-57a1a1977135 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e0c8d7aa4db455a94631becf212866f)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:25.721782 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e87030-4ad3-40b4-be54-57a1a1977135 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:25.727596 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:45:25.927460 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-75e87030-4ad3-40b4-be54-57a1a1977135 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e0c8d7aa4db455a94631becf212866f'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:25.970123 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-75e87030-4ad3-40b4-be54-57a1a1977135 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e0c8d7aa4db455a94631becf212866f'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:25.971001 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-75e87030-4ad3-40b4-be54-57a1a1977135 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e0c8d7aa4db455a94631becf212866f'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:25.971765 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-75e87030-4ad3-40b4-be54-57a1a1977135 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e0c8d7aa4db455a94631becf212866f'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:25.983255 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-75e87030-4ad3-40b4-be54-57a1a1977135 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e0c8d7aa4db455a94631becf212866f'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:26.005639 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1251/2492] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:45:25 2022] DELETE /identity/v3/users/5e0c8d7aa4db455a94631becf212866f => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 07:45:26.416693 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eaedd7-ec8f-4e20-adfd-77ce4d6a26c7 service placement] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:26.433061 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eaedd7-ec8f-4e20-adfd-77ce4d6a26c7 service placement] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:26.501509 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d1f486-95f2-46f5-843d-da482828a74a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:26.502221 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d1f486-95f2-46f5-843d-da482828a74a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:26.502425 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d1f486-95f2-46f5-843d-da482828a74a admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:26.502692 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d1f486-95f2-46f5-843d-da482828a74a admin admin] PATH_INFO: `/v3/projects/14017d5edbbd4068b529d5d92444182a` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:26.504368 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d1f486-95f2-46f5-843d-da482828a74a admin admin] RBAC: Authorizing `identity:delete_project(project_id=14017d5edbbd4068b529d5d92444182a)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:26.506350 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d1f486-95f2-46f5-843d-da482828a74a admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:26.610017 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-14d1f486-95f2-46f5-843d-da482828a74a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14017d5edbbd4068b529d5d92444182a', 'request_id': 'req-14d1f486-95f2-46f5-843d-da482828a74a'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:26.616535 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-14d1f486-95f2-46f5-843d-da482828a74a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14017d5edbbd4068b529d5d92444182a', 'request_id': 'req-14d1f486-95f2-46f5-843d-da482828a74a'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:26.665599 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-14d1f486-95f2-46f5-843d-da482828a74a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14017d5edbbd4068b529d5d92444182a', 'request_id': 'req-14d1f486-95f2-46f5-843d-da482828a74a'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:26.677399 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-14d1f486-95f2-46f5-843d-da482828a74a admin admin] The token cache is being invalidate because project 14017d5edbbd4068b529d5d92444182a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:45:26.677723 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-14d1f486-95f2-46f5-843d-da482828a74a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:26.678388 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-14d1f486-95f2-46f5-843d-da482828a74a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:26.679849 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1242/2493] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:45:26 2022] DELETE /identity/v3/projects/14017d5edbbd4068b529d5d92444182a => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:45:26.685935 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ee8803-f45c-47a2-85dd-096763ea6991 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:26.686249 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ee8803-f45c-47a2-85dd-096763ea6991 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:26.686531 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ee8803-f45c-47a2-85dd-096763ea6991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:26.708764 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-d9ee8803-f45c-47a2-85dd-096763ea6991 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:45:26.710039 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-d9ee8803-f45c-47a2-85dd-096763ea6991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:45:26.789433 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1252/2494] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:45:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 07:45:26.797392 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d7ae15-4939-411b-80ff-9f80f72cef7f None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:26.797815 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d7ae15-4939-411b-80ff-9f80f72cef7f None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:26.798162 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d7ae15-4939-411b-80ff-9f80f72cef7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:26.819879 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-86d7ae15-4939-411b-80ff-9f80f72cef7f None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:45:26.821185 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-86d7ae15-4939-411b-80ff-9f80f72cef7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:45:26.829642 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1243/2495] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:45:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 07:45:26.834685 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ee8803-f45c-47a2-85dd-096763ea6991 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:26.844734 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ee8803-f45c-47a2-85dd-096763ea6991 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:26.851604 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d973f34-7131-444f-83aa-05e740b77c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:26.852250 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d973f34-7131-444f-83aa-05e740b77c21 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:26.852535 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d973f34-7131-444f-83aa-05e740b77c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:26.852997 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d973f34-7131-444f-83aa-05e740b77c21 admin admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:26.853680 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d973f34-7131-444f-83aa-05e740b77c21 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:45:26.853989 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d973f34-7131-444f-83aa-05e740b77c21 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:26.855591 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d973f34-7131-444f-83aa-05e740b77c21 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:26.874126 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1253/2496] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 15 07:45:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:45:26.877258 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d7ae15-4939-411b-80ff-9f80f72cef7f None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:26.887008 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d7ae15-4939-411b-80ff-9f80f72cef7f None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:26.891608 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4402e68e-4ed0-431e-8c81-40c41a285c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:26.892230 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4402e68e-4ed0-431e-8c81-40c41a285c87 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:26.892619 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4402e68e-4ed0-431e-8c81-40c41a285c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:26.892893 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4402e68e-4ed0-431e-8c81-40c41a285c87 admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:26.893521 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4402e68e-4ed0-431e-8c81-40c41a285c87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:26.895724 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4402e68e-4ed0-431e-8c81-40c41a285c87 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:26.998491 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1244/2497] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:45:26 2022] POST /identity/v3/projects => generated 371 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:45:27.002897 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d973f34-7131-444f-83aa-05e740b77c21 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:27.023407 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d973f34-7131-444f-83aa-05e740b77c21 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:27.083886 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77e8c4e-c8c0-4658-b5e6-44ab13c1163a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:27.086235 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e8c4e-c8c0-4658-b5e6-44ab13c1163a admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:27.086235 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e8c4e-c8c0-4658-b5e6-44ab13c1163a admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:27.086235 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e8c4e-c8c0-4658-b5e6-44ab13c1163a admin admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:27.086235 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77e8c4e-c8c0-4658-b5e6-44ab13c1163a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:27.087645 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77e8c4e-c8c0-4658-b5e6-44ab13c1163a admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:27.180491 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1254/2498] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:45:27 2022] POST /identity/v3/users => generated 286 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 07:45:27.184831 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4402e68e-4ed0-431e-8c81-40c41a285c87 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:27.195516 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4402e68e-4ed0-431e-8c81-40c41a285c87 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:27.200242 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a27c4a7-c8bd-48f6-8e70-00d9914897b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:27.200946 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a27c4a7-c8bd-48f6-8e70-00d9914897b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:27.201222 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a27c4a7-c8bd-48f6-8e70-00d9914897b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:27.201486 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a27c4a7-c8bd-48f6-8e70-00d9914897b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:27.202193 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a27c4a7-c8bd-48f6-8e70-00d9914897b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:27.203785 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a27c4a7-c8bd-48f6-8e70-00d9914897b0 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:27.219806 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1245/2499] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:45:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:45:27.221117 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77e8c4e-c8c0-4658-b5e6-44ab13c1163a admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:27.232814 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:45:27.239502 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77e8c4e-c8c0-4658-b5e6-44ab13c1163a admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:27.244281 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2f218d-5cfb-47b4-82bc-33d04e264acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:27.244965 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2f218d-5cfb-47b4-82bc-33d04e264acc admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:27.245252 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2f218d-5cfb-47b4-82bc-33d04e264acc admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:27.245524 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2f218d-5cfb-47b4-82bc-33d04e264acc admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:27.246206 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2f218d-5cfb-47b4-82bc-33d04e264acc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:27.247813 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2f218d-5cfb-47b4-82bc-33d04e264acc admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:27.262046 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1255/2500] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:45:27 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:45:27.263209 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a27c4a7-c8bd-48f6-8e70-00d9914897b0 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:27.274213 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a27c4a7-c8bd-48f6-8e70-00d9914897b0 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:27.279141 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aafb06-8f54-433f-bdf8-78b6a151faee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:27.279813 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aafb06-8f54-433f-bdf8-78b6a151faee admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:27.280069 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aafb06-8f54-433f-bdf8-78b6a151faee admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:27.280344 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aafb06-8f54-433f-bdf8-78b6a151faee admin admin] PATH_INFO: `/v3/projects/9fddf1c148e44cfd963ec96b7eeb1941/users/8644032bf6d74229b6c5ba26bda51a94/roles/053e12af84e943e699460a9d76c2072b` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:27.298738 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89aafb06-8f54-433f-bdf8-78b6a151faee admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fddf1c148e44cfd963ec96b7eeb1941, user_id=8644032bf6d74229b6c5ba26bda51a94, role_id=053e12af84e943e699460a9d76c2072b)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:27.301107 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89aafb06-8f54-433f-bdf8-78b6a151faee admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:27.403996 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:45:27.405968 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1246/2501] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:45:27 2022] PUT /identity/v3/projects/9fddf1c148e44cfd963ec96b7eeb1941/users/8644032bf6d74229b6c5ba26bda51a94/roles/053e12af84e943e699460a9d76c2072b => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:45:27.412095 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7173f0-4719-46a0-a783-378fc00a77ee None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:27.412402 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7173f0-4719-46a0-a783-378fc00a77ee None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:27.412847 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7173f0-4719-46a0-a783-378fc00a77ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:27.457403 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-3e7173f0-4719-46a0-a783-378fc00a77ee None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:45:27.461146 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-3e7173f0-4719-46a0-a783-378fc00a77ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:45:27.527814 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1256/2502] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:45:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:45:27.538241 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aafb06-8f54-433f-bdf8-78b6a151faee admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:27.540167 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-89aafb06-8f54-433f-bdf8-78b6a151faee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:45:27.600505 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aafb06-8f54-433f-bdf8-78b6a151faee admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:27.605100 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d3cea0-6a60-47c8-ad3e-7f62d9a5e70c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:27.605744 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d3cea0-6a60-47c8-ad3e-7f62d9a5e70c service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:27.606013 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d3cea0-6a60-47c8-ad3e-7f62d9a5e70c service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:27.606297 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d3cea0-6a60-47c8-ad3e-7f62d9a5e70c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:27.616510 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d3cea0-6a60-47c8-ad3e-7f62d9a5e70c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:27.617990 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d3cea0-6a60-47c8-ad3e-7f62d9a5e70c service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:27.627662 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1247/2503] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:45:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:45:28.942658 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd8fc3c-f3df-4213-ab6f-064fd9784954 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:28.943337 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd8fc3c-f3df-4213-ab6f-064fd9784954 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:28.943337 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd8fc3c-f3df-4213-ab6f-064fd9784954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:28.975412 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-efd8fc3c-f3df-4213-ab6f-064fd9784954 None None] MFA Rules not processed for user `8644032bf6d74229b6c5ba26bda51a94`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:45:28.982362 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-efd8fc3c-f3df-4213-ab6f-064fd9784954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:45:29.060864 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1257/2504] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:45:28 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:45:29.068387 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d3cea0-6a60-47c8-ad3e-7f62d9a5e70c service neutron] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:29.077900 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d3cea0-6a60-47c8-ad3e-7f62d9a5e70c service neutron] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:29.084152 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d51033c-1677-41ef-b9fd-45575307cda9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:29.085023 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d51033c-1677-41ef-b9fd-45575307cda9 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:29.085309 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d51033c-1677-41ef-b9fd-45575307cda9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:29.085678 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d51033c-1677-41ef-b9fd-45575307cda9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:29.086645 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d51033c-1677-41ef-b9fd-45575307cda9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:29.088624 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d51033c-1677-41ef-b9fd-45575307cda9 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:29.214229 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1248/2505] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:45:29 2022] POST /identity/v3/projects => generated 373 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:45:29.215375 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd8fc3c-f3df-4213-ab6f-064fd9784954 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:29.229825 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd8fc3c-f3df-4213-ab6f-064fd9784954 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:29.307966 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc71fbf-33d9-4aaf-a933-9000288d032e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:29.308611 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc71fbf-33d9-4aaf-a933-9000288d032e admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:29.309890 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc71fbf-33d9-4aaf-a933-9000288d032e admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:29.310423 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc71fbf-33d9-4aaf-a933-9000288d032e admin admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:29.311147 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc71fbf-33d9-4aaf-a933-9000288d032e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:29.319327 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc71fbf-33d9-4aaf-a933-9000288d032e admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:29.419835 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1258/2506] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:45:29 2022] POST /identity/v3/users => generated 293 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 07:45:29.423917 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d51033c-1677-41ef-b9fd-45575307cda9 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:29.434877 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d51033c-1677-41ef-b9fd-45575307cda9 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:29.439559 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8fbd25-f114-4d64-abdc-2fdb927c1af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:29.440328 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8fbd25-f114-4d64-abdc-2fdb927c1af5 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:29.440621 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8fbd25-f114-4d64-abdc-2fdb927c1af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:29.440877 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8fbd25-f114-4d64-abdc-2fdb927c1af5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:29.441515 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8fbd25-f114-4d64-abdc-2fdb927c1af5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:29.443156 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8fbd25-f114-4d64-abdc-2fdb927c1af5 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:29.454130 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1249/2507] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:45:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:45:29.458938 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc71fbf-33d9-4aaf-a933-9000288d032e admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:29.475234 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc71fbf-33d9-4aaf-a933-9000288d032e admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:29.481787 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5405d50-8406-4831-b0e0-daecac6f6914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:29.481787 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5405d50-8406-4831-b0e0-daecac6f6914 admin admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:29.481787 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5405d50-8406-4831-b0e0-daecac6f6914 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:29.481787 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5405d50-8406-4831-b0e0-daecac6f6914 admin admin] PATH_INFO: `/v3/projects/d6fa330b75f34bb78b68c7847b96941b/users/deb2e2cafadb4f84b59bad8e79528487/roles/6bbb643e0d8d4bc0a9b27c26edf59216` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:29.491952 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:45:29.499135 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5405d50-8406-4831-b0e0-daecac6f6914 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6fa330b75f34bb78b68c7847b96941b, user_id=deb2e2cafadb4f84b59bad8e79528487, role_id=6bbb643e0d8d4bc0a9b27c26edf59216)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:29.503845 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5405d50-8406-4831-b0e0-daecac6f6914 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:29.608330 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1259/2508] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:45:29 2022] PUT /identity/v3/projects/d6fa330b75f34bb78b68c7847b96941b/users/deb2e2cafadb4f84b59bad8e79528487/roles/6bbb643e0d8d4bc0a9b27c26edf59216 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:45:29.612872 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8fbd25-f114-4d64-abdc-2fdb927c1af5 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:29.625503 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8fbd25-f114-4d64-abdc-2fdb927c1af5 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:29.700032 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf361c8e-749e-4171-80cc-96c6c383336b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:29.700637 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf361c8e-749e-4171-80cc-96c6c383336b admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:29.700917 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf361c8e-749e-4171-80cc-96c6c383336b admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:29.701189 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf361c8e-749e-4171-80cc-96c6c383336b admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:29.701863 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf361c8e-749e-4171-80cc-96c6c383336b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:29.703431 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf361c8e-749e-4171-80cc-96c6c383336b admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:29.717323 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1250/2509] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:45:29 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:45:29.718487 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5405d50-8406-4831-b0e0-daecac6f6914 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:29.728846 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5405d50-8406-4831-b0e0-daecac6f6914 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:29.733478 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0583f1-e38d-457d-a646-7bd086d3209d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:29.734174 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0583f1-e38d-457d-a646-7bd086d3209d admin admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:29.734444 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0583f1-e38d-457d-a646-7bd086d3209d admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:29.734712 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0583f1-e38d-457d-a646-7bd086d3209d admin admin] PATH_INFO: `/v3/projects/d6fa330b75f34bb78b68c7847b96941b/users/deb2e2cafadb4f84b59bad8e79528487/roles/6bbb643e0d8d4bc0a9b27c26edf59216` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:29.736918 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0583f1-e38d-457d-a646-7bd086d3209d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6fa330b75f34bb78b68c7847b96941b, user_id=deb2e2cafadb4f84b59bad8e79528487, role_id=6bbb643e0d8d4bc0a9b27c26edf59216)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:29.739624 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0583f1-e38d-457d-a646-7bd086d3209d admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:29.749911 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1260/2510] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:45:29 2022] PUT /identity/v3/projects/d6fa330b75f34bb78b68c7847b96941b/users/deb2e2cafadb4f84b59bad8e79528487/roles/6bbb643e0d8d4bc0a9b27c26edf59216 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:45:31.200214 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c9d219-2cec-4b64-9a28-3ac404dc3b05 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:31.201091 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c9d219-2cec-4b64-9a28-3ac404dc3b05 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:31.201360 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c9d219-2cec-4b64-9a28-3ac404dc3b05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:31.265357 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-c8c9d219-2cec-4b64-9a28-3ac404dc3b05 None None] MFA Rules not processed for user `deb2e2cafadb4f84b59bad8e79528487`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:45:31.267661 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-c8c9d219-2cec-4b64-9a28-3ac404dc3b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:45:31.364413 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:45:31.389163 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1251/2511] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:45:31 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:45:31.401899 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0583f1-e38d-457d-a646-7bd086d3209d admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:31.403661 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-0e0583f1-e38d-457d-a646-7bd086d3209d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:45:31.474599 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0583f1-e38d-457d-a646-7bd086d3209d admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:31.478851 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755a1ba0-5626-4137-b18b-f60455118c3a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecf6642b7b84d3c9756bb41ab3349d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:31.479470 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755a1ba0-5626-4137-b18b-f60455118c3a service cinder] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:31.479747 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755a1ba0-5626-4137-b18b-f60455118c3a service cinder] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:31.480025 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755a1ba0-5626-4137-b18b-f60455118c3a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:31.554898 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755a1ba0-5626-4137-b18b-f60455118c3a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:31.556328 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755a1ba0-5626-4137-b18b-f60455118c3a service cinder] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:31.565860 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1261/2512] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:45:31 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 07:45:33.371553 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c9d219-2cec-4b64-9a28-3ac404dc3b05 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:33.374131 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-c8c9d219-2cec-4b64-9a28-3ac404dc3b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:45:33.413691 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c9d219-2cec-4b64-9a28-3ac404dc3b05 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:33.428622 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c23f4d8-ffc9-4c1b-a2a9-e0a9d5d31564 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecf6642b7b84d3c9756bb41ab3349d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:33.429262 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c23f4d8-ffc9-4c1b-a2a9-e0a9d5d31564 service cinder] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:33.430111 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c23f4d8-ffc9-4c1b-a2a9-e0a9d5d31564 service cinder] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:33.430387 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c23f4d8-ffc9-4c1b-a2a9-e0a9d5d31564 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:33.442621 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c23f4d8-ffc9-4c1b-a2a9-e0a9d5d31564 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:33.444054 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c23f4d8-ffc9-4c1b-a2a9-e0a9d5d31564 service cinder] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:33.454491 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1252/2513] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:45:33 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:45:41.561297 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=58,keystone:INSERT=2 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:45:43.449539 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=10 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:45:44.612186 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755a1ba0-5626-4137-b18b-f60455118c3a service cinder] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:44.613745 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-755a1ba0-5626-4137-b18b-f60455118c3a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:45:44.696374 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755a1ba0-5626-4137-b18b-f60455118c3a service cinder] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:44.701943 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe837a94-b040-4b54-9b33-2762ce0a1dab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:44.702917 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe837a94-b040-4b54-9b33-2762ce0a1dab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:44.703437 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe837a94-b040-4b54-9b33-2762ce0a1dab admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:44.703744 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe837a94-b040-4b54-9b33-2762ce0a1dab admin admin] PATH_INFO: `/v3/users/28c5b285ac03442d856446e848ba895a` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:44.706028 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe837a94-b040-4b54-9b33-2762ce0a1dab admin admin] RBAC: Authorizing `identity:delete_user(user_id=28c5b285ac03442d856446e848ba895a)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:44.711102 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe837a94-b040-4b54-9b33-2762ce0a1dab admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:44.944936 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-fe837a94-b040-4b54-9b33-2762ce0a1dab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28c5b285ac03442d856446e848ba895a'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:45.022308 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-fe837a94-b040-4b54-9b33-2762ce0a1dab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28c5b285ac03442d856446e848ba895a'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:45.023355 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-fe837a94-b040-4b54-9b33-2762ce0a1dab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28c5b285ac03442d856446e848ba895a'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:45.026789 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-fe837a94-b040-4b54-9b33-2762ce0a1dab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28c5b285ac03442d856446e848ba895a'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:45.034930 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-fe837a94-b040-4b54-9b33-2762ce0a1dab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28c5b285ac03442d856446e848ba895a'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:45.073712 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1262/2514] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:45:44 2022] DELETE /identity/v3/users/28c5b285ac03442d856446e848ba895a => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:45:45.075393 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cfaed4-69c3-448e-b38e-bc7b469a14c7 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:45.075880 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cfaed4-69c3-448e-b38e-bc7b469a14c7 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:45.075979 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cfaed4-69c3-448e-b38e-bc7b469a14c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:45.100597 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-11cfaed4-69c3-448e-b38e-bc7b469a14c7 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:45:45.102121 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-11cfaed4-69c3-448e-b38e-bc7b469a14c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:45:45.187678 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1253/2515] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:45:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 07:45:45.197518 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe837a94-b040-4b54-9b33-2762ce0a1dab admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:45.201539 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-fe837a94-b040-4b54-9b33-2762ce0a1dab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:45:45.275496 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe837a94-b040-4b54-9b33-2762ce0a1dab admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:45.284106 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa4a70c-021b-418a-918c-1e9837f487d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:45.284922 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa4a70c-021b-418a-918c-1e9837f487d9 service neutron] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:45.285285 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa4a70c-021b-418a-918c-1e9837f487d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:45.285652 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa4a70c-021b-418a-918c-1e9837f487d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:45.297612 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa4a70c-021b-418a-918c-1e9837f487d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:45.299198 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa4a70c-021b-418a-918c-1e9837f487d9 service neutron] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:45.312447 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1263/2516] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:45:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 07:45:45.760036 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cfaed4-69c3-448e-b38e-bc7b469a14c7 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:45.762608 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-11cfaed4-69c3-448e-b38e-bc7b469a14c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:45:45.781334 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cfaed4-69c3-448e-b38e-bc7b469a14c7 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:45.792235 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8094ca0-e017-425c-8e81-4f3cb094faa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:45.792882 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8094ca0-e017-425c-8e81-4f3cb094faa8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:45.793162 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8094ca0-e017-425c-8e81-4f3cb094faa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:45.793433 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8094ca0-e017-425c-8e81-4f3cb094faa8 admin admin] PATH_INFO: `/v3/projects/6a599d6dd6a34afabf5a156ba7f3981e` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:45.794609 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8094ca0-e017-425c-8e81-4f3cb094faa8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a599d6dd6a34afabf5a156ba7f3981e)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:45.796550 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8094ca0-e017-425c-8e81-4f3cb094faa8 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:45.919088 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-d8094ca0-e017-425c-8e81-4f3cb094faa8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a599d6dd6a34afabf5a156ba7f3981e', 'request_id': 'req-d8094ca0-e017-425c-8e81-4f3cb094faa8'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:45.930105 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-d8094ca0-e017-425c-8e81-4f3cb094faa8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a599d6dd6a34afabf5a156ba7f3981e', 'request_id': 'req-d8094ca0-e017-425c-8e81-4f3cb094faa8'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:45.993105 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-d8094ca0-e017-425c-8e81-4f3cb094faa8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a599d6dd6a34afabf5a156ba7f3981e', 'request_id': 'req-d8094ca0-e017-425c-8e81-4f3cb094faa8'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:46.005486 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-d8094ca0-e017-425c-8e81-4f3cb094faa8 admin admin] The token cache is being invalidate because project 6a599d6dd6a34afabf5a156ba7f3981e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:45:46.006296 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-d8094ca0-e017-425c-8e81-4f3cb094faa8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:46.009611 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-d8094ca0-e017-425c-8e81-4f3cb094faa8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:46.011908 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1254/2517] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:45:45 2022] DELETE /identity/v3/projects/6a599d6dd6a34afabf5a156ba7f3981e => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:45:46.016860 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208d301f-80ae-4afb-9d48-2b674a703c77 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:46.017487 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208d301f-80ae-4afb-9d48-2b674a703c77 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:46.017487 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208d301f-80ae-4afb-9d48-2b674a703c77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:46.038738 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-208d301f-80ae-4afb-9d48-2b674a703c77 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:45:46.038951 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-208d301f-80ae-4afb-9d48-2b674a703c77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:45:46.066243 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:45:46.122763 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1264/2518] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:45:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:45:46.136735 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851c60b8-5ba1-464e-a76e-c8b777f6ecb6 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:46.137232 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851c60b8-5ba1-464e-a76e-c8b777f6ecb6 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:46.141616 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851c60b8-5ba1-464e-a76e-c8b777f6ecb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:46.181360 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-851c60b8-5ba1-464e-a76e-c8b777f6ecb6 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:45:46.182636 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-851c60b8-5ba1-464e-a76e-c8b777f6ecb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:45:46.198644 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1255/2519] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:45:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:45:46.203185 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208d301f-80ae-4afb-9d48-2b674a703c77 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:46.233764 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208d301f-80ae-4afb-9d48-2b674a703c77 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:46.249013 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879f9149-94fa-4d89-bff8-064027959d99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:46.249379 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879f9149-94fa-4d89-bff8-064027959d99 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:46.249797 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879f9149-94fa-4d89-bff8-064027959d99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:46.250335 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879f9149-94fa-4d89-bff8-064027959d99 admin admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:46.251673 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879f9149-94fa-4d89-bff8-064027959d99 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:45:46.253485 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879f9149-94fa-4d89-bff8-064027959d99 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:46.256473 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879f9149-94fa-4d89-bff8-064027959d99 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:46.275602 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1265/2520] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 15 07:45:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:45:46.280338 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851c60b8-5ba1-464e-a76e-c8b777f6ecb6 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:46.291202 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851c60b8-5ba1-464e-a76e-c8b777f6ecb6 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:46.296086 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfbf7d8-c9a1-470f-93e4-594228ad4cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:46.302457 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfbf7d8-c9a1-470f-93e4-594228ad4cad admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:46.302457 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfbf7d8-c9a1-470f-93e4-594228ad4cad admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:46.302457 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfbf7d8-c9a1-470f-93e4-594228ad4cad admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:46.302457 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfbf7d8-c9a1-470f-93e4-594228ad4cad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:46.302457 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfbf7d8-c9a1-470f-93e4-594228ad4cad admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:46.390231 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1256/2521] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:45:46 2022] POST /identity/v3/projects => generated 377 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:45:46.394873 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879f9149-94fa-4d89-bff8-064027959d99 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:46.406172 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879f9149-94fa-4d89-bff8-064027959d99 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:46.471682 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5643aef2-44f6-41f6-a055-718f659b6789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:46.472455 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5643aef2-44f6-41f6-a055-718f659b6789 admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:46.472895 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5643aef2-44f6-41f6-a055-718f659b6789 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:46.473242 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5643aef2-44f6-41f6-a055-718f659b6789 admin admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:46.473982 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5643aef2-44f6-41f6-a055-718f659b6789 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:46.476081 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5643aef2-44f6-41f6-a055-718f659b6789 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:46.587776 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1266/2522] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:45:46 2022] POST /identity/v3/users => generated 289 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 07:45:46.591945 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfbf7d8-c9a1-470f-93e4-594228ad4cad admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:46.602830 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfbf7d8-c9a1-470f-93e4-594228ad4cad admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:46.608500 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcaa948-a67b-4eef-ba83-41f9c3c75ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:46.608966 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcaa948-a67b-4eef-ba83-41f9c3c75ac9 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:46.609341 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcaa948-a67b-4eef-ba83-41f9c3c75ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:46.609836 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcaa948-a67b-4eef-ba83-41f9c3c75ac9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:46.610880 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcaa948-a67b-4eef-ba83-41f9c3c75ac9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:46.612511 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcaa948-a67b-4eef-ba83-41f9c3c75ac9 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:46.622639 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1257/2523] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:45:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:45:46.626826 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5643aef2-44f6-41f6-a055-718f659b6789 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:46.636740 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5643aef2-44f6-41f6-a055-718f659b6789 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:46.641906 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848d5cda-106f-48bb-b3b8-d7f88e7793f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:46.642501 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d5cda-106f-48bb-b3b8-d7f88e7793f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:46.642783 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d5cda-106f-48bb-b3b8-d7f88e7793f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:46.643129 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d5cda-106f-48bb-b3b8-d7f88e7793f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:46.643769 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848d5cda-106f-48bb-b3b8-d7f88e7793f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:46.645361 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848d5cda-106f-48bb-b3b8-d7f88e7793f2 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:46.656247 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1267/2524] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:45:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:45:46.665881 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcaa948-a67b-4eef-ba83-41f9c3c75ac9 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:46.671125 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcaa948-a67b-4eef-ba83-41f9c3c75ac9 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:46.675757 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2240ddbd-ec00-44e0-91b0-88251a1045ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:46.676395 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2240ddbd-ec00-44e0-91b0-88251a1045ae admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:46.676672 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2240ddbd-ec00-44e0-91b0-88251a1045ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:46.676942 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2240ddbd-ec00-44e0-91b0-88251a1045ae admin admin] PATH_INFO: `/v3/projects/99c0ca510d39467a8fd3a9ad77926902/users/871dc078665a4cdbaee351b7b4205009/roles/053e12af84e943e699460a9d76c2072b` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:46.691169 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2240ddbd-ec00-44e0-91b0-88251a1045ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=99c0ca510d39467a8fd3a9ad77926902, user_id=871dc078665a4cdbaee351b7b4205009, role_id=053e12af84e943e699460a9d76c2072b)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:46.693842 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2240ddbd-ec00-44e0-91b0-88251a1045ae admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:46.795592 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1258/2525] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:45:46 2022] PUT /identity/v3/projects/99c0ca510d39467a8fd3a9ad77926902/users/871dc078665a4cdbaee351b7b4205009/roles/053e12af84e943e699460a9d76c2072b => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:45:46.805899 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe074ccb-c1b3-45f5-a973-f2843dc3d22a None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:46.806187 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe074ccb-c1b3-45f5-a973-f2843dc3d22a None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:46.806469 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe074ccb-c1b3-45f5-a973-f2843dc3d22a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:46.844746 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-fe074ccb-c1b3-45f5-a973-f2843dc3d22a None None] MFA Rules not processed for user `871dc078665a4cdbaee351b7b4205009`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:45:46.846354 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-fe074ccb-c1b3-45f5-a973-f2843dc3d22a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:45:46.911300 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1268/2526] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:45:46 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:45:46.922926 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2240ddbd-ec00-44e0-91b0-88251a1045ae admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:46.924817 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-2240ddbd-ec00-44e0-91b0-88251a1045ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:45:46.948910 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:45:46.984779 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2240ddbd-ec00-44e0-91b0-88251a1045ae admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:46.989024 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5648755-490b-4311-a602-3cb7cc4614cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecf6642b7b84d3c9756bb41ab3349d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:46.989656 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5648755-490b-4311-a602-3cb7cc4614cd service cinder] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:46.989935 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5648755-490b-4311-a602-3cb7cc4614cd service cinder] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:46.990206 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5648755-490b-4311-a602-3cb7cc4614cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:46.998355 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5648755-490b-4311-a602-3cb7cc4614cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:46.999750 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5648755-490b-4311-a602-3cb7cc4614cd service cinder] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:47.009551 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1259/2527] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:45:46 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:45:49.307619 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe074ccb-c1b3-45f5-a973-f2843dc3d22a None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:49.309052 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-fe074ccb-c1b3-45f5-a973-f2843dc3d22a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:45:49.330730 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:45:49.376296 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe074ccb-c1b3-45f5-a973-f2843dc3d22a None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:49.384790 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f892cd7-cff7-436f-9c1d-31abd0d0501f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:49.387757 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f892cd7-cff7-436f-9c1d-31abd0d0501f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:49.387757 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f892cd7-cff7-436f-9c1d-31abd0d0501f admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:49.387757 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f892cd7-cff7-436f-9c1d-31abd0d0501f admin admin] PATH_INFO: `/v3/users/8644032bf6d74229b6c5ba26bda51a94` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:49.387757 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f892cd7-cff7-436f-9c1d-31abd0d0501f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8644032bf6d74229b6c5ba26bda51a94)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:49.389331 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f892cd7-cff7-436f-9c1d-31abd0d0501f admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:49.565954 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-8f892cd7-cff7-436f-9c1d-31abd0d0501f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8644032bf6d74229b6c5ba26bda51a94'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:49.655240 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-8f892cd7-cff7-436f-9c1d-31abd0d0501f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8644032bf6d74229b6c5ba26bda51a94'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:49.655905 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-8f892cd7-cff7-436f-9c1d-31abd0d0501f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8644032bf6d74229b6c5ba26bda51a94'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:49.656723 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-8f892cd7-cff7-436f-9c1d-31abd0d0501f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8644032bf6d74229b6c5ba26bda51a94'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:49.664970 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-8f892cd7-cff7-436f-9c1d-31abd0d0501f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8644032bf6d74229b6c5ba26bda51a94'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:49.683857 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1269/2528] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:45:49 2022] DELETE /identity/v3/users/8644032bf6d74229b6c5ba26bda51a94 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:45:49.914271 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5648755-490b-4311-a602-3cb7cc4614cd service cinder] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:49.928006 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5648755-490b-4311-a602-3cb7cc4614cd service cinder] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:49.991105 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d91ecb-72e3-49ca-854a-d49c1977288c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:49.991732 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d91ecb-72e3-49ca-854a-d49c1977288c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:49.992019 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d91ecb-72e3-49ca-854a-d49c1977288c admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:49.992297 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d91ecb-72e3-49ca-854a-d49c1977288c admin admin] PATH_INFO: `/v3/projects/9fddf1c148e44cfd963ec96b7eeb1941` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:49.993408 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d91ecb-72e3-49ca-854a-d49c1977288c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fddf1c148e44cfd963ec96b7eeb1941)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:49.995350 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d91ecb-72e3-49ca-854a-d49c1977288c admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:50.107948 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-d4d91ecb-72e3-49ca-854a-d49c1977288c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fddf1c148e44cfd963ec96b7eeb1941', 'request_id': 'req-d4d91ecb-72e3-49ca-854a-d49c1977288c'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:50.114807 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-d4d91ecb-72e3-49ca-854a-d49c1977288c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fddf1c148e44cfd963ec96b7eeb1941', 'request_id': 'req-d4d91ecb-72e3-49ca-854a-d49c1977288c'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:50.198346 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-d4d91ecb-72e3-49ca-854a-d49c1977288c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fddf1c148e44cfd963ec96b7eeb1941', 'request_id': 'req-d4d91ecb-72e3-49ca-854a-d49c1977288c'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:50.210248 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-d4d91ecb-72e3-49ca-854a-d49c1977288c admin admin] The token cache is being invalidate because project 9fddf1c148e44cfd963ec96b7eeb1941 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:45:50.210544 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-d4d91ecb-72e3-49ca-854a-d49c1977288c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:50.211172 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-d4d91ecb-72e3-49ca-854a-d49c1977288c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:50.212657 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1260/2529] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:45:49 2022] DELETE /identity/v3/projects/9fddf1c148e44cfd963ec96b7eeb1941 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:45:50.216618 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f892cd7-cff7-436f-9c1d-31abd0d0501f admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:50.218052 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-8f892cd7-cff7-436f-9c1d-31abd0d0501f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:45:50.231185 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f892cd7-cff7-436f-9c1d-31abd0d0501f admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:50.291197 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce10565-da08-4d53-a25e-8aa53ced5565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:50.291830 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce10565-da08-4d53-a25e-8aa53ced5565 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:50.292107 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce10565-da08-4d53-a25e-8aa53ced5565 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:50.292389 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce10565-da08-4d53-a25e-8aa53ced5565 admin admin] PATH_INFO: `/v3/users/deb2e2cafadb4f84b59bad8e79528487` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:50.293507 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce10565-da08-4d53-a25e-8aa53ced5565 admin admin] RBAC: Authorizing `identity:delete_user(user_id=deb2e2cafadb4f84b59bad8e79528487)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:50.295459 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce10565-da08-4d53-a25e-8aa53ced5565 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:50.492803 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:45:50.498557 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-bce10565-da08-4d53-a25e-8aa53ced5565 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'deb2e2cafadb4f84b59bad8e79528487'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:50.615321 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-bce10565-da08-4d53-a25e-8aa53ced5565 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'deb2e2cafadb4f84b59bad8e79528487'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:50.615940 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-bce10565-da08-4d53-a25e-8aa53ced5565 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'deb2e2cafadb4f84b59bad8e79528487'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:50.616741 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-bce10565-da08-4d53-a25e-8aa53ced5565 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'deb2e2cafadb4f84b59bad8e79528487'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:50.624481 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-bce10565-da08-4d53-a25e-8aa53ced5565 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'deb2e2cafadb4f84b59bad8e79528487'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:50.642609 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1270/2530] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:45:50 2022] DELETE /identity/v3/users/deb2e2cafadb4f84b59bad8e79528487 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:45:50.884937 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d91ecb-72e3-49ca-854a-d49c1977288c admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:50.886590 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-d4d91ecb-72e3-49ca-854a-d49c1977288c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:45:50.900460 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d91ecb-72e3-49ca-854a-d49c1977288c admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:50.950905 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:45:50.982439 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814a8f49-5d62-40ad-9004-76d75e332541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:50.982439 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814a8f49-5d62-40ad-9004-76d75e332541 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:50.982439 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814a8f49-5d62-40ad-9004-76d75e332541 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:50.982439 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814a8f49-5d62-40ad-9004-76d75e332541 admin admin] PATH_INFO: `/v3/projects/d6fa330b75f34bb78b68c7847b96941b` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:50.982879 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814a8f49-5d62-40ad-9004-76d75e332541 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6fa330b75f34bb78b68c7847b96941b)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:50.982879 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814a8f49-5d62-40ad-9004-76d75e332541 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:51.096316 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-814a8f49-5d62-40ad-9004-76d75e332541 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6fa330b75f34bb78b68c7847b96941b', 'request_id': 'req-814a8f49-5d62-40ad-9004-76d75e332541'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:51.102366 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-814a8f49-5d62-40ad-9004-76d75e332541 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6fa330b75f34bb78b68c7847b96941b', 'request_id': 'req-814a8f49-5d62-40ad-9004-76d75e332541'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:51.158240 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-814a8f49-5d62-40ad-9004-76d75e332541 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6fa330b75f34bb78b68c7847b96941b', 'request_id': 'req-814a8f49-5d62-40ad-9004-76d75e332541'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:51.170128 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-814a8f49-5d62-40ad-9004-76d75e332541 admin admin] The token cache is being invalidate because project d6fa330b75f34bb78b68c7847b96941b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:45:51.170452 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-814a8f49-5d62-40ad-9004-76d75e332541 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:51.171109 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-814a8f49-5d62-40ad-9004-76d75e332541 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:51.179512 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1261/2531] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:45:50 2022] DELETE /identity/v3/projects/d6fa330b75f34bb78b68c7847b96941b => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 07:45:51.179634 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de19feb-5d0d-4035-a32e-bc5f9f0c5ed7 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:51.179634 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de19feb-5d0d-4035-a32e-bc5f9f0c5ed7 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:51.179634 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de19feb-5d0d-4035-a32e-bc5f9f0c5ed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:51.205655 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-9de19feb-5d0d-4035-a32e-bc5f9f0c5ed7 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:45:51.206921 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-9de19feb-5d0d-4035-a32e-bc5f9f0c5ed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:45:51.273720 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1271/2532] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:45:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 07:45:51.281691 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b93184-a47d-45ce-9ab8-502ee6d14295 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:51.282162 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b93184-a47d-45ce-9ab8-502ee6d14295 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:51.282442 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b93184-a47d-45ce-9ab8-502ee6d14295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:51.302882 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-77b93184-a47d-45ce-9ab8-502ee6d14295 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:45:51.304081 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-77b93184-a47d-45ce-9ab8-502ee6d14295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:45:51.311843 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1262/2533] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:45:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:45:51.316792 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de19feb-5d0d-4035-a32e-bc5f9f0c5ed7 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:51.326439 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de19feb-5d0d-4035-a32e-bc5f9f0c5ed7 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:51.331149 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00c19f6-a626-4397-a83a-c9e6578877ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:51.336872 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00c19f6-a626-4397-a83a-c9e6578877ac admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:51.336872 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00c19f6-a626-4397-a83a-c9e6578877ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:51.336872 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00c19f6-a626-4397-a83a-c9e6578877ac admin admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:51.336872 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00c19f6-a626-4397-a83a-c9e6578877ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:45:51.336872 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00c19f6-a626-4397-a83a-c9e6578877ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:51.336872 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00c19f6-a626-4397-a83a-c9e6578877ac admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:51.347229 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1272/2534] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 15 07:45:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:45:51.351456 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b93184-a47d-45ce-9ab8-502ee6d14295 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:51.361730 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b93184-a47d-45ce-9ab8-502ee6d14295 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:51.366368 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adacd11d-1156-430b-bc0a-733d68001362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:51.366960 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adacd11d-1156-430b-bc0a-733d68001362 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:51.367241 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adacd11d-1156-430b-bc0a-733d68001362 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:51.367514 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adacd11d-1156-430b-bc0a-733d68001362 admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:51.368138 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adacd11d-1156-430b-bc0a-733d68001362 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:51.370051 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adacd11d-1156-430b-bc0a-733d68001362 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:51.462156 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1263/2535] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:45:51 2022] POST /identity/v3/projects => generated 385 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:45:51.466394 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00c19f6-a626-4397-a83a-c9e6578877ac admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:51.476619 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00c19f6-a626-4397-a83a-c9e6578877ac admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:51.541143 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e9fec6-8836-4e63-809c-d618b9772f78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:51.541828 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e9fec6-8836-4e63-809c-d618b9772f78 admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:51.542063 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e9fec6-8836-4e63-809c-d618b9772f78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:51.542339 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e9fec6-8836-4e63-809c-d618b9772f78 admin admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:51.542972 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e9fec6-8836-4e63-809c-d618b9772f78 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:51.544899 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e9fec6-8836-4e63-809c-d618b9772f78 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:51.603406 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1273/2536] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:45:51 2022] POST /identity/v3/users => generated 293 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:45:51.607901 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adacd11d-1156-430b-bc0a-733d68001362 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:51.618557 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adacd11d-1156-430b-bc0a-733d68001362 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:51.623206 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44681098-d056-4320-9d6d-fa95ba37be33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:51.623798 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44681098-d056-4320-9d6d-fa95ba37be33 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:51.624077 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44681098-d056-4320-9d6d-fa95ba37be33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:51.624350 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44681098-d056-4320-9d6d-fa95ba37be33 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:51.625675 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44681098-d056-4320-9d6d-fa95ba37be33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:51.627417 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44681098-d056-4320-9d6d-fa95ba37be33 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:51.637557 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1264/2537] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:45:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:45:51.642360 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e9fec6-8836-4e63-809c-d618b9772f78 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:51.652154 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e9fec6-8836-4e63-809c-d618b9772f78 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:51.657493 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fe7fde-1e59-4635-9f04-7055e8c507c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:51.658110 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe7fde-1e59-4635-9f04-7055e8c507c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:51.658390 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe7fde-1e59-4635-9f04-7055e8c507c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:51.658670 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe7fde-1e59-4635-9f04-7055e8c507c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:51.659298 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fe7fde-1e59-4635-9f04-7055e8c507c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:51.660891 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fe7fde-1e59-4635-9f04-7055e8c507c0 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:51.673082 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1274/2538] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:45:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 07:45:51.677189 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44681098-d056-4320-9d6d-fa95ba37be33 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:51.687032 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44681098-d056-4320-9d6d-fa95ba37be33 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:51.691731 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f63236-04d5-4f20-be2e-477d137c9371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:51.692371 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f63236-04d5-4f20-be2e-477d137c9371 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:51.692650 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f63236-04d5-4f20-be2e-477d137c9371 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:51.692927 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f63236-04d5-4f20-be2e-477d137c9371 admin admin] PATH_INFO: `/v3/projects/0b6b9cbe658645a0b02febf7cd46c695/users/bd8b5574732146b9b62d37904c3c13fe/roles/053e12af84e943e699460a9d76c2072b` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:51.708219 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f63236-04d5-4f20-be2e-477d137c9371 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b6b9cbe658645a0b02febf7cd46c695, user_id=bd8b5574732146b9b62d37904c3c13fe, role_id=053e12af84e943e699460a9d76c2072b)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:51.710942 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f63236-04d5-4f20-be2e-477d137c9371 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:51.809377 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1265/2539] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:45:51 2022] PUT /identity/v3/projects/0b6b9cbe658645a0b02febf7cd46c695/users/bd8b5574732146b9b62d37904c3c13fe/roles/053e12af84e943e699460a9d76c2072b => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 07:45:51.823312 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728fd09c-911f-4c38-ac22-9acd715d4105 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:51.823641 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728fd09c-911f-4c38-ac22-9acd715d4105 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:51.823914 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728fd09c-911f-4c38-ac22-9acd715d4105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:51.876935 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-728fd09c-911f-4c38-ac22-9acd715d4105 None None] MFA Rules not processed for user `bd8b5574732146b9b62d37904c3c13fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:45:51.878123 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-728fd09c-911f-4c38-ac22-9acd715d4105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:45:51.887386 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:45:51.943910 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1275/2540] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:45:51 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:45:51.948367 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f63236-04d5-4f20-be2e-477d137c9371 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:52.011369 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f63236-04d5-4f20-be2e-477d137c9371 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:52.015883 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d58b203-0b34-4189-9a11-1c9b97a6b24f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:52.016475 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d58b203-0b34-4189-9a11-1c9b97a6b24f admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:52.016753 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d58b203-0b34-4189-9a11-1c9b97a6b24f admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:52.017022 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d58b203-0b34-4189-9a11-1c9b97a6b24f admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:52.017667 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d58b203-0b34-4189-9a11-1c9b97a6b24f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:52.019525 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d58b203-0b34-4189-9a11-1c9b97a6b24f admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:52.114454 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1266/2541] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:45:51 2022] POST /identity/v3/projects => generated 385 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:45:52.122457 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728fd09c-911f-4c38-ac22-9acd715d4105 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:52.133508 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728fd09c-911f-4c38-ac22-9acd715d4105 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:52.193693 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c50997-d551-4ab7-9e2d-b03dbb0fa562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:52.194313 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c50997-d551-4ab7-9e2d-b03dbb0fa562 admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:52.194597 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c50997-d551-4ab7-9e2d-b03dbb0fa562 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:52.194876 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c50997-d551-4ab7-9e2d-b03dbb0fa562 admin admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:52.195522 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c50997-d551-4ab7-9e2d-b03dbb0fa562 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:52.197441 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c50997-d551-4ab7-9e2d-b03dbb0fa562 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:52.277890 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1276/2542] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:45:52 2022] POST /identity/v3/users => generated 299 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:45:52.283331 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d58b203-0b34-4189-9a11-1c9b97a6b24f admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:52.290035 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:45:52.293839 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d58b203-0b34-4189-9a11-1c9b97a6b24f admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:52.299196 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b2404f-1f94-4d2c-9176-4f93186e6362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:52.299823 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b2404f-1f94-4d2c-9176-4f93186e6362 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:52.300754 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b2404f-1f94-4d2c-9176-4f93186e6362 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:52.300754 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b2404f-1f94-4d2c-9176-4f93186e6362 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:52.300973 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b2404f-1f94-4d2c-9176-4f93186e6362 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:52.302710 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b2404f-1f94-4d2c-9176-4f93186e6362 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:52.313217 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1267/2543] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:45:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:45:52.317495 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c50997-d551-4ab7-9e2d-b03dbb0fa562 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:52.327498 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c50997-d551-4ab7-9e2d-b03dbb0fa562 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:52.332261 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab041fa7-0190-40e2-b3d4-9902a5c6a56c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:52.332912 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab041fa7-0190-40e2-b3d4-9902a5c6a56c admin admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:52.333197 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab041fa7-0190-40e2-b3d4-9902a5c6a56c admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:52.335152 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab041fa7-0190-40e2-b3d4-9902a5c6a56c admin admin] PATH_INFO: `/v3/projects/1e4bf6cae5844e948f69618537148255/users/14961bc48d6c473ca1c9a4422637577a/roles/6bbb643e0d8d4bc0a9b27c26edf59216` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:52.349619 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab041fa7-0190-40e2-b3d4-9902a5c6a56c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e4bf6cae5844e948f69618537148255, user_id=14961bc48d6c473ca1c9a4422637577a, role_id=6bbb643e0d8d4bc0a9b27c26edf59216)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:52.352385 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab041fa7-0190-40e2-b3d4-9902a5c6a56c admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:52.420501 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1277/2544] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:45:52 2022] PUT /identity/v3/projects/1e4bf6cae5844e948f69618537148255/users/14961bc48d6c473ca1c9a4422637577a/roles/6bbb643e0d8d4bc0a9b27c26edf59216 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 07:45:52.425294 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b2404f-1f94-4d2c-9176-4f93186e6362 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:52.435507 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b2404f-1f94-4d2c-9176-4f93186e6362 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:52.493980 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e03522-eba9-4434-b542-8d7ed5b8ff05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:52.494586 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e03522-eba9-4434-b542-8d7ed5b8ff05 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:52.494869 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e03522-eba9-4434-b542-8d7ed5b8ff05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:52.495140 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e03522-eba9-4434-b542-8d7ed5b8ff05 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:52.495775 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e03522-eba9-4434-b542-8d7ed5b8ff05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:52.497355 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e03522-eba9-4434-b542-8d7ed5b8ff05 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:52.507319 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1268/2545] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:45:52 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:45:52.511613 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab041fa7-0190-40e2-b3d4-9902a5c6a56c admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:52.522853 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab041fa7-0190-40e2-b3d4-9902a5c6a56c admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:52.527655 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92b166d-bec2-41e8-9570-f2003ffbfd81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:52.528433 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92b166d-bec2-41e8-9570-f2003ffbfd81 admin admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:52.528723 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92b166d-bec2-41e8-9570-f2003ffbfd81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:52.529419 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92b166d-bec2-41e8-9570-f2003ffbfd81 admin admin] PATH_INFO: `/v3/projects/1e4bf6cae5844e948f69618537148255/users/14961bc48d6c473ca1c9a4422637577a/roles/6bbb643e0d8d4bc0a9b27c26edf59216` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:52.531137 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92b166d-bec2-41e8-9570-f2003ffbfd81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e4bf6cae5844e948f69618537148255, user_id=14961bc48d6c473ca1c9a4422637577a, role_id=6bbb643e0d8d4bc0a9b27c26edf59216)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:52.534138 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92b166d-bec2-41e8-9570-f2003ffbfd81 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:52.545314 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1278/2546] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:45:52 2022] PUT /identity/v3/projects/1e4bf6cae5844e948f69618537148255/users/14961bc48d6c473ca1c9a4422637577a/roles/6bbb643e0d8d4bc0a9b27c26edf59216 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:45:52.556402 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e7b9f4-d1a1-4c0d-a997-efb6f768c156 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:52.556727 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e7b9f4-d1a1-4c0d-a997-efb6f768c156 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:52.557006 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e7b9f4-d1a1-4c0d-a997-efb6f768c156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:52.585332 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-b0e7b9f4-d1a1-4c0d-a997-efb6f768c156 None None] MFA Rules not processed for user `14961bc48d6c473ca1c9a4422637577a`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:45:52.586609 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-b0e7b9f4-d1a1-4c0d-a997-efb6f768c156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:45:52.659828 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1269/2547] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:45:52 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:45:52.670595 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92b166d-bec2-41e8-9570-f2003ffbfd81 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:52.672529 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-d92b166d-bec2-41e8-9570-f2003ffbfd81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:45:52.732878 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92b166d-bec2-41e8-9570-f2003ffbfd81 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:52.736698 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba1556a-2179-4e37-bc51-6734772ea0d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecf6642b7b84d3c9756bb41ab3349d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:52.737303 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba1556a-2179-4e37-bc51-6734772ea0d8 service cinder] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:52.737594 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba1556a-2179-4e37-bc51-6734772ea0d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:52.737890 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba1556a-2179-4e37-bc51-6734772ea0d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:52.758732 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:45:52.799851 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba1556a-2179-4e37-bc51-6734772ea0d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:52.801305 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba1556a-2179-4e37-bc51-6734772ea0d8 service cinder] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:52.810923 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1279/2548] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:45:52 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:45:52.891771 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e7b9f4-d1a1-4c0d-a997-efb6f768c156 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:52.893209 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-b0e7b9f4-d1a1-4c0d-a997-efb6f768c156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:45:52.951399 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e7b9f4-d1a1-4c0d-a997-efb6f768c156 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:52.955226 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5533c892-6444-41a6-b577-713bfc9a8d08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:52.955840 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5533c892-6444-41a6-b577-713bfc9a8d08 service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:52.956228 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5533c892-6444-41a6-b577-713bfc9a8d08 service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:52.956383 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5533c892-6444-41a6-b577-713bfc9a8d08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:52.958141 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-5533c892-6444-41a6-b577-713bfc9a8d08 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:45:52.990039 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:45:53.024133 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5533c892-6444-41a6-b577-713bfc9a8d08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:53.025562 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5533c892-6444-41a6-b577-713bfc9a8d08 service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:53.035936 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1270/2549] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:45:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:45:56.844328 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba1556a-2179-4e37-bc51-6734772ea0d8 service cinder] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:56.845787 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-aba1556a-2179-4e37-bc51-6734772ea0d8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:45:56.863190 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba1556a-2179-4e37-bc51-6734772ea0d8 service cinder] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:56.867866 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d382d73e-1406-4662-9a33-cb72a11e825a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:56.868525 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d382d73e-1406-4662-9a33-cb72a11e825a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:56.868814 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d382d73e-1406-4662-9a33-cb72a11e825a admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:56.869086 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d382d73e-1406-4662-9a33-cb72a11e825a admin admin] PATH_INFO: `/v3/users/56c2f854cf074b1ba2571766b7f2fc51` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:56.870367 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d382d73e-1406-4662-9a33-cb72a11e825a admin admin] RBAC: Authorizing `identity:delete_user(user_id=56c2f854cf074b1ba2571766b7f2fc51)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:56.872329 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d382d73e-1406-4662-9a33-cb72a11e825a admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:57.044777 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-d382d73e-1406-4662-9a33-cb72a11e825a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56c2f854cf074b1ba2571766b7f2fc51'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:57.132981 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-d382d73e-1406-4662-9a33-cb72a11e825a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56c2f854cf074b1ba2571766b7f2fc51'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:57.133638 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-d382d73e-1406-4662-9a33-cb72a11e825a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56c2f854cf074b1ba2571766b7f2fc51'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:57.134243 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-d382d73e-1406-4662-9a33-cb72a11e825a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56c2f854cf074b1ba2571766b7f2fc51'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:57.141810 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-d382d73e-1406-4662-9a33-cb72a11e825a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56c2f854cf074b1ba2571766b7f2fc51'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:57.159288 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1280/2550] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:45:56 2022] DELETE /identity/v3/users/56c2f854cf074b1ba2571766b7f2fc51 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 07:45:57.517851 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5533c892-6444-41a6-b577-713bfc9a8d08 service neutron] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:57.528003 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5533c892-6444-41a6-b577-713bfc9a8d08 service neutron] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:57.587914 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a21dd2-ae94-4e94-81be-f1fb1aa4f472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:57.588543 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a21dd2-ae94-4e94-81be-f1fb1aa4f472 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:57.588822 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a21dd2-ae94-4e94-81be-f1fb1aa4f472 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:57.589123 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a21dd2-ae94-4e94-81be-f1fb1aa4f472 admin admin] PATH_INFO: `/v3/projects/b4a8f2664964492e9c80a595ed043c71` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:57.590244 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a21dd2-ae94-4e94-81be-f1fb1aa4f472 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4a8f2664964492e9c80a595ed043c71)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:57.592203 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a21dd2-ae94-4e94-81be-f1fb1aa4f472 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:57.725862 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-f9a21dd2-ae94-4e94-81be-f1fb1aa4f472 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4a8f2664964492e9c80a595ed043c71', 'request_id': 'req-f9a21dd2-ae94-4e94-81be-f1fb1aa4f472'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:57.732966 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-f9a21dd2-ae94-4e94-81be-f1fb1aa4f472 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4a8f2664964492e9c80a595ed043c71', 'request_id': 'req-f9a21dd2-ae94-4e94-81be-f1fb1aa4f472'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:57.785207 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-f9a21dd2-ae94-4e94-81be-f1fb1aa4f472 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4a8f2664964492e9c80a595ed043c71', 'request_id': 'req-f9a21dd2-ae94-4e94-81be-f1fb1aa4f472'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:57.797333 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-f9a21dd2-ae94-4e94-81be-f1fb1aa4f472 admin admin] The token cache is being invalidate because project b4a8f2664964492e9c80a595ed043c71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:45:57.797741 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-f9a21dd2-ae94-4e94-81be-f1fb1aa4f472 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:57.798469 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-f9a21dd2-ae94-4e94-81be-f1fb1aa4f472 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:45:57.800305 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1271/2551] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:45:57 2022] DELETE /identity/v3/projects/b4a8f2664964492e9c80a595ed043c71 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:45:57.805162 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952c4002-0e10-4418-ac97-c286c99f907d None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:57.805453 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952c4002-0e10-4418-ac97-c286c99f907d None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:57.805776 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952c4002-0e10-4418-ac97-c286c99f907d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:57.826154 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-952c4002-0e10-4418-ac97-c286c99f907d None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:45:57.827354 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-952c4002-0e10-4418-ac97-c286c99f907d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:45:57.914898 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1281/2552] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:45:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:45:57.922882 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fab3655-0b59-47d3-96c6-839453abb2fb None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:57.923103 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fab3655-0b59-47d3-96c6-839453abb2fb None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:57.923361 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fab3655-0b59-47d3-96c6-839453abb2fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:57.944846 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-7fab3655-0b59-47d3-96c6-839453abb2fb None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:45:57.946252 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-7fab3655-0b59-47d3-96c6-839453abb2fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:45:57.958899 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1272/2553] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:45:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 07:45:57.980791 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952c4002-0e10-4418-ac97-c286c99f907d None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:57.980791 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952c4002-0e10-4418-ac97-c286c99f907d None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:57.980791 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60726cc9-524e-44ae-b459-da8ac706f26c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:57.980791 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60726cc9-524e-44ae-b459-da8ac706f26c admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:57.982141 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60726cc9-524e-44ae-b459-da8ac706f26c admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:57.982141 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60726cc9-524e-44ae-b459-da8ac706f26c admin admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:57.982141 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60726cc9-524e-44ae-b459-da8ac706f26c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:45:57.982141 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60726cc9-524e-44ae-b459-da8ac706f26c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:57.984388 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60726cc9-524e-44ae-b459-da8ac706f26c admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:57.996155 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1282/2554] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 15 07:45:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:45:58.001013 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fab3655-0b59-47d3-96c6-839453abb2fb None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:58.010424 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fab3655-0b59-47d3-96c6-839453abb2fb None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:58.016533 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450c1848-9c41-4fb0-9b0b-8cc9c4a0bee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:58.017304 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450c1848-9c41-4fb0-9b0b-8cc9c4a0bee7 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:58.017612 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450c1848-9c41-4fb0-9b0b-8cc9c4a0bee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:58.018406 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450c1848-9c41-4fb0-9b0b-8cc9c4a0bee7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:58.018518 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450c1848-9c41-4fb0-9b0b-8cc9c4a0bee7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:58.020562 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450c1848-9c41-4fb0-9b0b-8cc9c4a0bee7 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:58.085277 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1273/2555] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:45:58 2022] POST /identity/v3/projects => generated 377 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:45:58.089362 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60726cc9-524e-44ae-b459-da8ac706f26c admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:58.101809 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60726cc9-524e-44ae-b459-da8ac706f26c admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:58.114977 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:45:58.161538 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f711eb5-ba4c-47bb-8606-8f78e27bc3b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:58.162181 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f711eb5-ba4c-47bb-8606-8f78e27bc3b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:58.162504 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f711eb5-ba4c-47bb-8606-8f78e27bc3b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:58.162804 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f711eb5-ba4c-47bb-8606-8f78e27bc3b0 admin admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:58.163447 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f711eb5-ba4c-47bb-8606-8f78e27bc3b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:58.165372 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f711eb5-ba4c-47bb-8606-8f78e27bc3b0 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:58.271364 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1283/2556] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:45:58 2022] POST /identity/v3/users => generated 289 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:45:58.275639 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450c1848-9c41-4fb0-9b0b-8cc9c4a0bee7 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:58.286773 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450c1848-9c41-4fb0-9b0b-8cc9c4a0bee7 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:58.291416 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177b3fd9-fdc2-4116-8414-b2e64a8fbc02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:58.292012 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177b3fd9-fdc2-4116-8414-b2e64a8fbc02 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:58.292357 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177b3fd9-fdc2-4116-8414-b2e64a8fbc02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:58.292553 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177b3fd9-fdc2-4116-8414-b2e64a8fbc02 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:58.293214 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177b3fd9-fdc2-4116-8414-b2e64a8fbc02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:58.294825 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177b3fd9-fdc2-4116-8414-b2e64a8fbc02 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:58.305367 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1274/2557] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:45:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:45:58.309393 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f711eb5-ba4c-47bb-8606-8f78e27bc3b0 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:58.319263 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f711eb5-ba4c-47bb-8606-8f78e27bc3b0 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:58.323889 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b955065-0aba-4c43-821d-00c9926016d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:58.324478 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b955065-0aba-4c43-821d-00c9926016d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:58.324758 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b955065-0aba-4c43-821d-00c9926016d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:58.325030 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b955065-0aba-4c43-821d-00c9926016d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:58.325690 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b955065-0aba-4c43-821d-00c9926016d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:58.327280 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b955065-0aba-4c43-821d-00c9926016d1 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:58.338779 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1284/2558] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:45:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:45:58.342838 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177b3fd9-fdc2-4116-8414-b2e64a8fbc02 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:58.353686 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177b3fd9-fdc2-4116-8414-b2e64a8fbc02 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:58.358599 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18542c45-cc5b-43fa-9696-3ad3f6a0c001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:58.359978 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18542c45-cc5b-43fa-9696-3ad3f6a0c001 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:58.360293 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18542c45-cc5b-43fa-9696-3ad3f6a0c001 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:58.360594 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18542c45-cc5b-43fa-9696-3ad3f6a0c001 admin admin] PATH_INFO: `/v3/projects/a9d0f5e6d6cf423485157e1381a12713/users/f8022d5f904e4ae294b2cdf21aa11212/roles/053e12af84e943e699460a9d76c2072b` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:58.373497 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:45:58.376575 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18542c45-cc5b-43fa-9696-3ad3f6a0c001 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9d0f5e6d6cf423485157e1381a12713, user_id=f8022d5f904e4ae294b2cdf21aa11212, role_id=053e12af84e943e699460a9d76c2072b)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:58.379367 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18542c45-cc5b-43fa-9696-3ad3f6a0c001 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:58.446070 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1275/2559] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:45:58 2022] PUT /identity/v3/projects/a9d0f5e6d6cf423485157e1381a12713/users/f8022d5f904e4ae294b2cdf21aa11212/roles/053e12af84e943e699460a9d76c2072b => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:45:58.449399 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6634e3e-61b5-42f5-96ca-85df14279d7e None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:58.449752 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6634e3e-61b5-42f5-96ca-85df14279d7e None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:58.450028 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6634e3e-61b5-42f5-96ca-85df14279d7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:58.471179 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-c6634e3e-61b5-42f5-96ca-85df14279d7e None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:45:58.475412 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-c6634e3e-61b5-42f5-96ca-85df14279d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:45:58.539623 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1285/2560] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:45:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:45:58.549755 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18542c45-cc5b-43fa-9696-3ad3f6a0c001 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:45:58.551509 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-18542c45-cc5b-43fa-9696-3ad3f6a0c001 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:45:58.611675 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18542c45-cc5b-43fa-9696-3ad3f6a0c001 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:45:58.615286 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f52c88-0703-42d0-a6bb-ce0e973f607a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:45:58.615886 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f52c88-0703-42d0-a6bb-ce0e973f607a service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:45:58.616162 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f52c88-0703-42d0-a6bb-ce0e973f607a service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:45:58.616431 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f52c88-0703-42d0-a6bb-ce0e973f607a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:45:58.625306 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f52c88-0703-42d0-a6bb-ce0e973f607a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:45:58.629717 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f52c88-0703-42d0-a6bb-ce0e973f607a service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:45:58.635845 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1276/2561] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:45:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:46:01.287969 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6634e3e-61b5-42f5-96ca-85df14279d7e None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:01.289352 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-c6634e3e-61b5-42f5-96ca-85df14279d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:46:01.349125 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6634e3e-61b5-42f5-96ca-85df14279d7e None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:01.352969 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed043f6-0a3a-406e-9705-e20003ce081f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecf6642b7b84d3c9756bb41ab3349d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:01.353621 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed043f6-0a3a-406e-9705-e20003ce081f service cinder] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:01.353911 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed043f6-0a3a-406e-9705-e20003ce081f service cinder] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:01.354188 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed043f6-0a3a-406e-9705-e20003ce081f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:01.356333 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-9ed043f6-0a3a-406e-9705-e20003ce081f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:46:01.422450 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:46:01.438859 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed043f6-0a3a-406e-9705-e20003ce081f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:01.440456 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed043f6-0a3a-406e-9705-e20003ce081f service cinder] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:01.452987 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1286/2562] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:46:01 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:46:04.161138 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1d15e4-8dd3-48da-8c62-1f61c68acd63 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:04.161668 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1d15e4-8dd3-48da-8c62-1f61c68acd63 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:04.161797 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1d15e4-8dd3-48da-8c62-1f61c68acd63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:04.193354 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-ab1d15e4-8dd3-48da-8c62-1f61c68acd63 None None] MFA Rules not processed for user `f8022d5f904e4ae294b2cdf21aa11212`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:46:04.194659 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-ab1d15e4-8dd3-48da-8c62-1f61c68acd63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:46:04.282046 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1277/2563] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:46:04 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:46:04.290305 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed043f6-0a3a-406e-9705-e20003ce081f service cinder] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:04.291802 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-9ed043f6-0a3a-406e-9705-e20003ce081f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:46:04.378279 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed043f6-0a3a-406e-9705-e20003ce081f service cinder] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:04.384247 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfb1a48-1ab6-49ec-9597-e288c6526d79 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a080c7ab9b684abca6828067f2572c3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:04.384891 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfb1a48-1ab6-49ec-9597-e288c6526d79 service nova] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:04.385172 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfb1a48-1ab6-49ec-9597-e288c6526d79 service nova] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:04.385447 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfb1a48-1ab6-49ec-9597-e288c6526d79 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:04.395985 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfb1a48-1ab6-49ec-9597-e288c6526d79 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:04.397692 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfb1a48-1ab6-49ec-9597-e288c6526d79 service nova] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:04.414351 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1287/2564] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:46:04 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:46:06.889083 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2614d2ab-ccd2-4456-be56-1992015f8640 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:06.889624 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2614d2ab-ccd2-4456-be56-1992015f8640 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:06.889740 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2614d2ab-ccd2-4456-be56-1992015f8640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:06.917767 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-2614d2ab-ccd2-4456-be56-1992015f8640 None None] MFA Rules not processed for user `f8022d5f904e4ae294b2cdf21aa11212`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:46:06.922254 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-2614d2ab-ccd2-4456-be56-1992015f8640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:46:06.929094 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1278/2565] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:46:06 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:46:08.535036 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfb1a48-1ab6-49ec-9597-e288c6526d79 service nova] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:08.536451 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-5bfb1a48-1ab6-49ec-9597-e288c6526d79 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:46:08.549857 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfb1a48-1ab6-49ec-9597-e288c6526d79 service nova] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:08.554306 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae2221d-6a5e-41f5-ba59-1af74c42502f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:08.554928 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae2221d-6a5e-41f5-ba59-1af74c42502f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:08.555209 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae2221d-6a5e-41f5-ba59-1af74c42502f admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:08.555484 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae2221d-6a5e-41f5-ba59-1af74c42502f admin admin] PATH_INFO: `/v3/users/bd8b5574732146b9b62d37904c3c13fe` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:08.556587 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae2221d-6a5e-41f5-ba59-1af74c42502f admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd8b5574732146b9b62d37904c3c13fe)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:08.558538 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae2221d-6a5e-41f5-ba59-1af74c42502f admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:08.854862 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-aae2221d-6a5e-41f5-ba59-1af74c42502f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd8b5574732146b9b62d37904c3c13fe'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:08.938196 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-aae2221d-6a5e-41f5-ba59-1af74c42502f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd8b5574732146b9b62d37904c3c13fe'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:08.938829 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-aae2221d-6a5e-41f5-ba59-1af74c42502f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd8b5574732146b9b62d37904c3c13fe'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:08.939447 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-aae2221d-6a5e-41f5-ba59-1af74c42502f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd8b5574732146b9b62d37904c3c13fe'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:08.947687 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-aae2221d-6a5e-41f5-ba59-1af74c42502f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd8b5574732146b9b62d37904c3c13fe'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:08.965689 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1288/2566] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:46:08 2022] DELETE /identity/v3/users/bd8b5574732146b9b62d37904c3c13fe => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 07:46:08.970334 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004912f1-ca0e-48ad-9df9-2e865556f2be None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:08.970889 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004912f1-ca0e-48ad-9df9-2e865556f2be None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:08.982616 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004912f1-ca0e-48ad-9df9-2e865556f2be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:08.990983 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-004912f1-ca0e-48ad-9df9-2e865556f2be None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:46:08.992185 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-004912f1-ca0e-48ad-9df9-2e865556f2be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:46:09.052104 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1279/2567] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:46:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:46:09.063164 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae2221d-6a5e-41f5-ba59-1af74c42502f admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:09.064638 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-aae2221d-6a5e-41f5-ba59-1af74c42502f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:46:09.111868 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:46:09.122959 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae2221d-6a5e-41f5-ba59-1af74c42502f admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:09.126676 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e90a58-57c4-46cd-8ce4-a0a1c3be73cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:09.127286 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e90a58-57c4-46cd-8ce4-a0a1c3be73cc service neutron] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:09.127564 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e90a58-57c4-46cd-8ce4-a0a1c3be73cc service neutron] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:09.127834 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e90a58-57c4-46cd-8ce4-a0a1c3be73cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:09.137449 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e90a58-57c4-46cd-8ce4-a0a1c3be73cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:09.139577 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e90a58-57c4-46cd-8ce4-a0a1c3be73cc service neutron] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:09.149058 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1289/2568] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:46:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:46:09.739763 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004912f1-ca0e-48ad-9df9-2e865556f2be None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:09.741439 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-004912f1-ca0e-48ad-9df9-2e865556f2be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:46:09.755144 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004912f1-ca0e-48ad-9df9-2e865556f2be None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:09.760000 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a5e333-3e0c-4e40-b448-6bd92c8cf7e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:09.760655 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a5e333-3e0c-4e40-b448-6bd92c8cf7e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:09.760923 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a5e333-3e0c-4e40-b448-6bd92c8cf7e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:09.761195 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a5e333-3e0c-4e40-b448-6bd92c8cf7e8 admin admin] PATH_INFO: `/v3/projects/0b6b9cbe658645a0b02febf7cd46c695` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:09.762583 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a5e333-3e0c-4e40-b448-6bd92c8cf7e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b6b9cbe658645a0b02febf7cd46c695)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:09.764923 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a5e333-3e0c-4e40-b448-6bd92c8cf7e8 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:09.782857 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:46:09.905384 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-c6a5e333-3e0c-4e40-b448-6bd92c8cf7e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b6b9cbe658645a0b02febf7cd46c695', 'request_id': 'req-c6a5e333-3e0c-4e40-b448-6bd92c8cf7e8'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:09.912055 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-c6a5e333-3e0c-4e40-b448-6bd92c8cf7e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b6b9cbe658645a0b02febf7cd46c695', 'request_id': 'req-c6a5e333-3e0c-4e40-b448-6bd92c8cf7e8'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:09.996597 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-c6a5e333-3e0c-4e40-b448-6bd92c8cf7e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b6b9cbe658645a0b02febf7cd46c695', 'request_id': 'req-c6a5e333-3e0c-4e40-b448-6bd92c8cf7e8'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:10.008594 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-c6a5e333-3e0c-4e40-b448-6bd92c8cf7e8 admin admin] The token cache is being invalidate because project 0b6b9cbe658645a0b02febf7cd46c695 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:46:10.008956 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-c6a5e333-3e0c-4e40-b448-6bd92c8cf7e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:10.009650 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-c6a5e333-3e0c-4e40-b448-6bd92c8cf7e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:10.011191 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1280/2569] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:46:09 2022] DELETE /identity/v3/projects/0b6b9cbe658645a0b02febf7cd46c695 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 07:46:10.015411 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e90a58-57c4-46cd-8ce4-a0a1c3be73cc service neutron] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:10.025620 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e90a58-57c4-46cd-8ce4-a0a1c3be73cc service neutron] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:10.098315 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbf1c4e-acff-436d-8845-bf75e2b17f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:10.099053 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbf1c4e-acff-436d-8845-bf75e2b17f81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:10.099400 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbf1c4e-acff-436d-8845-bf75e2b17f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:10.099742 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbf1c4e-acff-436d-8845-bf75e2b17f81 admin admin] PATH_INFO: `/v3/users/14961bc48d6c473ca1c9a4422637577a` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:10.100937 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbf1c4e-acff-436d-8845-bf75e2b17f81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=14961bc48d6c473ca1c9a4422637577a)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:10.103054 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbf1c4e-acff-436d-8845-bf75e2b17f81 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:10.340971 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-4cbf1c4e-acff-436d-8845-bf75e2b17f81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14961bc48d6c473ca1c9a4422637577a'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:10.434503 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-4cbf1c4e-acff-436d-8845-bf75e2b17f81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14961bc48d6c473ca1c9a4422637577a'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:10.434503 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-4cbf1c4e-acff-436d-8845-bf75e2b17f81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14961bc48d6c473ca1c9a4422637577a'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:10.435760 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-4cbf1c4e-acff-436d-8845-bf75e2b17f81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14961bc48d6c473ca1c9a4422637577a'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:10.443868 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-4cbf1c4e-acff-436d-8845-bf75e2b17f81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14961bc48d6c473ca1c9a4422637577a'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:10.463675 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1290/2570] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:46:10 2022] DELETE /identity/v3/users/14961bc48d6c473ca1c9a4422637577a => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:46:11.004018 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a5e333-3e0c-4e40-b448-6bd92c8cf7e8 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:11.006015 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-c6a5e333-3e0c-4e40-b448-6bd92c8cf7e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:46:11.026433 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a5e333-3e0c-4e40-b448-6bd92c8cf7e8 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:11.102858 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e907414-4cff-4bda-9cbb-7da9fd030787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:11.103492 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e907414-4cff-4bda-9cbb-7da9fd030787 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:11.103995 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e907414-4cff-4bda-9cbb-7da9fd030787 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:11.104680 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e907414-4cff-4bda-9cbb-7da9fd030787 admin admin] PATH_INFO: `/v3/projects/1e4bf6cae5844e948f69618537148255` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:11.105995 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e907414-4cff-4bda-9cbb-7da9fd030787 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e4bf6cae5844e948f69618537148255)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:11.108465 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e907414-4cff-4bda-9cbb-7da9fd030787 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:11.227798 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-4e907414-4cff-4bda-9cbb-7da9fd030787 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e4bf6cae5844e948f69618537148255', 'request_id': 'req-4e907414-4cff-4bda-9cbb-7da9fd030787'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:11.237180 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-4e907414-4cff-4bda-9cbb-7da9fd030787 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e4bf6cae5844e948f69618537148255', 'request_id': 'req-4e907414-4cff-4bda-9cbb-7da9fd030787'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:11.302935 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-4e907414-4cff-4bda-9cbb-7da9fd030787 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e4bf6cae5844e948f69618537148255', 'request_id': 'req-4e907414-4cff-4bda-9cbb-7da9fd030787'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:11.314202 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbf1c4e-acff-436d-8845-bf75e2b17f81 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:11.316138 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-4cbf1c4e-acff-436d-8845-bf75e2b17f81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:46:11.318157 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-4e907414-4cff-4bda-9cbb-7da9fd030787 admin admin] The token cache is being invalidate because project 1e4bf6cae5844e948f69618537148255 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:46:11.318447 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-4e907414-4cff-4bda-9cbb-7da9fd030787 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:11.319224 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-4e907414-4cff-4bda-9cbb-7da9fd030787 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:11.321124 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1281/2571] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:46:11 2022] DELETE /identity/v3/projects/1e4bf6cae5844e948f69618537148255 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:46:11.327039 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0f660a-2b06-4d7e-a003-a615f0f6dfe5 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:11.327329 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0f660a-2b06-4d7e-a003-a615f0f6dfe5 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:11.327610 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0f660a-2b06-4d7e-a003-a615f0f6dfe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:11.329099 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbf1c4e-acff-436d-8845-bf75e2b17f81 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:11.348381 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-3e0f660a-2b06-4d7e-a003-a615f0f6dfe5 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:46:11.349894 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-3e0f660a-2b06-4d7e-a003-a615f0f6dfe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:46:11.383952 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:46:11.391933 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed7a75b-ab00-47b9-9040-4968d43230c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:11.403069 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed7a75b-ab00-47b9-9040-4968d43230c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:11.403069 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed7a75b-ab00-47b9-9040-4968d43230c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:11.403069 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed7a75b-ab00-47b9-9040-4968d43230c5 admin admin] PATH_INFO: `/v3/users/871dc078665a4cdbaee351b7b4205009` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:11.403069 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed7a75b-ab00-47b9-9040-4968d43230c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=871dc078665a4cdbaee351b7b4205009)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:11.403069 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed7a75b-ab00-47b9-9040-4968d43230c5 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:11.405148 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:46:11.427482 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1282/2572] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:46:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 07:46:11.435811 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce8c330-789e-42da-90cb-ef8dda8c59b4 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:11.436098 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce8c330-789e-42da-90cb-ef8dda8c59b4 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:11.436401 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce8c330-789e-42da-90cb-ef8dda8c59b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:11.458386 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-4ce8c330-789e-42da-90cb-ef8dda8c59b4 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:46:11.459594 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-4ce8c330-789e-42da-90cb-ef8dda8c59b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:46:11.469483 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1283/2573] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:46:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 07:46:11.473870 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce8c330-789e-42da-90cb-ef8dda8c59b4 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:11.482931 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce8c330-789e-42da-90cb-ef8dda8c59b4 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:11.487510 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8499b499-fe6d-403f-b38b-a512d62899a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:11.488081 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8499b499-fe6d-403f-b38b-a512d62899a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:11.491466 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8499b499-fe6d-403f-b38b-a512d62899a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:11.491466 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8499b499-fe6d-403f-b38b-a512d62899a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:11.491466 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8499b499-fe6d-403f-b38b-a512d62899a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:46:11.491466 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8499b499-fe6d-403f-b38b-a512d62899a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:11.491466 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8499b499-fe6d-403f-b38b-a512d62899a4 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:11.502325 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1284/2574] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 15 07:46:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:46:11.506529 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8499b499-fe6d-403f-b38b-a512d62899a4 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:11.516196 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8499b499-fe6d-403f-b38b-a512d62899a4 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:11.520715 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6931ab7b-b4a4-49f1-b2fd-fc04e994c0c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:11.521279 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6931ab7b-b4a4-49f1-b2fd-fc04e994c0c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:11.521557 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6931ab7b-b4a4-49f1-b2fd-fc04e994c0c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:11.521859 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6931ab7b-b4a4-49f1-b2fd-fc04e994c0c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:11.522475 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6931ab7b-b4a4-49f1-b2fd-fc04e994c0c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:11.524302 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6931ab7b-b4a4-49f1-b2fd-fc04e994c0c7 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:11.563422 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-9ed7a75b-ab00-47b9-9040-4968d43230c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '871dc078665a4cdbaee351b7b4205009'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:11.620104 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1285/2575] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:46:11 2022] POST /identity/v3/projects => generated 385 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:46:11.624410 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6931ab7b-b4a4-49f1-b2fd-fc04e994c0c7 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:11.629153 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-9ed7a75b-ab00-47b9-9040-4968d43230c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '871dc078665a4cdbaee351b7b4205009'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:11.629913 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-9ed7a75b-ab00-47b9-9040-4968d43230c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '871dc078665a4cdbaee351b7b4205009'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:11.630619 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-9ed7a75b-ab00-47b9-9040-4968d43230c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '871dc078665a4cdbaee351b7b4205009'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:11.640101 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-9ed7a75b-ab00-47b9-9040-4968d43230c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '871dc078665a4cdbaee351b7b4205009'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:11.663676 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1291/2576] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:46:11 2022] DELETE /identity/v3/users/871dc078665a4cdbaee351b7b4205009 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 07:46:11.668413 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a12f97-5d16-4dfe-aa69-d7482af753cf None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:11.668945 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a12f97-5d16-4dfe-aa69-d7482af753cf None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:11.669273 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a12f97-5d16-4dfe-aa69-d7482af753cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:11.692269 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-c4a12f97-5d16-4dfe-aa69-d7482af753cf None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:46:11.693604 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-c4a12f97-5d16-4dfe-aa69-d7482af753cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:46:11.703547 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6931ab7b-b4a4-49f1-b2fd-fc04e994c0c7 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:11.765041 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1292/2577] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:46:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:46:11.774488 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1766bd31-0b1f-4e59-9b07-29158b76fa36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:11.775223 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1766bd31-0b1f-4e59-9b07-29158b76fa36 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:11.775544 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1766bd31-0b1f-4e59-9b07-29158b76fa36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:11.776013 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1766bd31-0b1f-4e59-9b07-29158b76fa36 admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:11.776598 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a12f97-5d16-4dfe-aa69-d7482af753cf None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:11.776741 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1766bd31-0b1f-4e59-9b07-29158b76fa36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:11.778272 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-c4a12f97-5d16-4dfe-aa69-d7482af753cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:46:11.779769 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1766bd31-0b1f-4e59-9b07-29158b76fa36 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:11.842280 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a12f97-5d16-4dfe-aa69-d7482af753cf None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:11.846050 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3491a473-e876-4ef5-a4c6-ad3d368e935a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:11.846661 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3491a473-e876-4ef5-a4c6-ad3d368e935a service neutron] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:11.846943 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3491a473-e876-4ef5-a4c6-ad3d368e935a service neutron] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:11.847217 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3491a473-e876-4ef5-a4c6-ad3d368e935a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:11.857902 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3491a473-e876-4ef5-a4c6-ad3d368e935a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:11.861699 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1286/2578] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:46:11 2022] POST /identity/v3/users => generated 293 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 07:46:11.862115 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3491a473-e876-4ef5-a4c6-ad3d368e935a service neutron] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:11.863008 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1766bd31-0b1f-4e59-9b07-29158b76fa36 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:11.864781 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-1766bd31-0b1f-4e59-9b07-29158b76fa36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:46:11.870495 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1293/2579] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:46:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 07:46:11.877705 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1766bd31-0b1f-4e59-9b07-29158b76fa36 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:11.882358 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c945ea37-14f6-437c-b118-8e8f4e05db83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:11.884237 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c945ea37-14f6-437c-b118-8e8f4e05db83 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:11.884237 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c945ea37-14f6-437c-b118-8e8f4e05db83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:11.884237 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c945ea37-14f6-437c-b118-8e8f4e05db83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:11.884237 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c945ea37-14f6-437c-b118-8e8f4e05db83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:11.885739 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c945ea37-14f6-437c-b118-8e8f4e05db83 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:11.897059 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1287/2580] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:46:11 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:46:11.901070 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3491a473-e876-4ef5-a4c6-ad3d368e935a service neutron] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:11.911034 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3491a473-e876-4ef5-a4c6-ad3d368e935a service neutron] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:11.915796 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd0c2a2-99ea-4a67-aa4f-59e3412add1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:11.916396 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd0c2a2-99ea-4a67-aa4f-59e3412add1c admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:11.916675 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd0c2a2-99ea-4a67-aa4f-59e3412add1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:11.916948 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd0c2a2-99ea-4a67-aa4f-59e3412add1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:11.917785 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd0c2a2-99ea-4a67-aa4f-59e3412add1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:11.919759 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd0c2a2-99ea-4a67-aa4f-59e3412add1c admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:11.930701 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1294/2581] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:46:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 07:46:11.935889 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c945ea37-14f6-437c-b118-8e8f4e05db83 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:11.946003 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c945ea37-14f6-437c-b118-8e8f4e05db83 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:11.951687 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd39019-bf17-49b4-9fce-fb73a262de82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:11.952568 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd39019-bf17-49b4-9fce-fb73a262de82 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:11.952689 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd39019-bf17-49b4-9fce-fb73a262de82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:11.953075 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd39019-bf17-49b4-9fce-fb73a262de82 admin admin] PATH_INFO: `/v3/projects/87211e3d93d54012a968a182217df89f/users/8db58b5e74574338ba8c16adc52e9c43/roles/053e12af84e943e699460a9d76c2072b` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:11.969167 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:46:11.972676 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd39019-bf17-49b4-9fce-fb73a262de82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87211e3d93d54012a968a182217df89f, user_id=8db58b5e74574338ba8c16adc52e9c43, role_id=053e12af84e943e699460a9d76c2072b)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:11.975417 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd39019-bf17-49b4-9fce-fb73a262de82 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:12.047845 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1288/2582] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:46:11 2022] PUT /identity/v3/projects/87211e3d93d54012a968a182217df89f/users/8db58b5e74574338ba8c16adc52e9c43/roles/053e12af84e943e699460a9d76c2072b => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 07:46:12.060091 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bb56b4-050e-454c-8a03-38fa91159eb9 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:12.060373 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bb56b4-050e-454c-8a03-38fa91159eb9 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:12.060649 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bb56b4-050e-454c-8a03-38fa91159eb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:12.090599 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-a9bb56b4-050e-454c-8a03-38fa91159eb9 None None] MFA Rules not processed for user `8db58b5e74574338ba8c16adc52e9c43`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:46:12.092511 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-a9bb56b4-050e-454c-8a03-38fa91159eb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:46:12.097807 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:46:12.157886 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1295/2583] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:46:12 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:46:12.162312 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd39019-bf17-49b4-9fce-fb73a262de82 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:12.249344 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd39019-bf17-49b4-9fce-fb73a262de82 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:12.254409 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcddf76-ecd9-4943-9d32-621551d83b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:12.255030 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcddf76-ecd9-4943-9d32-621551d83b79 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:12.255314 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcddf76-ecd9-4943-9d32-621551d83b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:12.255600 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcddf76-ecd9-4943-9d32-621551d83b79 admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:12.256250 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcddf76-ecd9-4943-9d32-621551d83b79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:12.258197 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcddf76-ecd9-4943-9d32-621551d83b79 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:12.369963 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1289/2584] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:46:12 2022] POST /identity/v3/projects => generated 385 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:46:12.374268 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bb56b4-050e-454c-8a03-38fa91159eb9 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:12.385503 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bb56b4-050e-454c-8a03-38fa91159eb9 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:12.396666 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcddf76-ecd9-4943-9d32-621551d83b79 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:12.399871 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-dbcddf76-ecd9-4943-9d32-621551d83b79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:46:12.504483 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcddf76-ecd9-4943-9d32-621551d83b79 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:12.509267 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69783ca-9e78-4a0b-ae40-27bba7a9da39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:12.509995 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69783ca-9e78-4a0b-ae40-27bba7a9da39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:12.510273 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69783ca-9e78-4a0b-ae40-27bba7a9da39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:12.510545 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69783ca-9e78-4a0b-ae40-27bba7a9da39 admin admin] PATH_INFO: `/v3/projects/99c0ca510d39467a8fd3a9ad77926902` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:12.511727 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69783ca-9e78-4a0b-ae40-27bba7a9da39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99c0ca510d39467a8fd3a9ad77926902)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:12.514334 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69783ca-9e78-4a0b-ae40-27bba7a9da39 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:12.514844 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335cc28c-f75b-4886-8fa2-f32f4e42a294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:12.515668 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335cc28c-f75b-4886-8fa2-f32f4e42a294 admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:12.516065 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335cc28c-f75b-4886-8fa2-f32f4e42a294 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:12.516341 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335cc28c-f75b-4886-8fa2-f32f4e42a294 admin admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:12.517114 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335cc28c-f75b-4886-8fa2-f32f4e42a294 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:12.519199 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335cc28c-f75b-4886-8fa2-f32f4e42a294 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:12.637403 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1296/2585] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:46:12 2022] POST /identity/v3/users => generated 293 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:46:12.639157 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335cc28c-f75b-4886-8fa2-f32f4e42a294 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:12.649171 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335cc28c-f75b-4886-8fa2-f32f4e42a294 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:12.655154 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1567c4b8-ec67-47e4-9922-1b6cbd488fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:12.655750 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1567c4b8-ec67-47e4-9922-1b6cbd488fc0 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:12.656030 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1567c4b8-ec67-47e4-9922-1b6cbd488fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:12.656299 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1567c4b8-ec67-47e4-9922-1b6cbd488fc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:12.656935 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1567c4b8-ec67-47e4-9922-1b6cbd488fc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:12.658554 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1567c4b8-ec67-47e4-9922-1b6cbd488fc0 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:12.670909 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1297/2586] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:46:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 07:46:12.674869 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1567c4b8-ec67-47e4-9922-1b6cbd488fc0 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:12.685469 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-e69783ca-9e78-4a0b-ae40-27bba7a9da39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99c0ca510d39467a8fd3a9ad77926902', 'request_id': 'req-e69783ca-9e78-4a0b-ae40-27bba7a9da39'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:12.687566 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1567c4b8-ec67-47e4-9922-1b6cbd488fc0 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:12.693396 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-e69783ca-9e78-4a0b-ae40-27bba7a9da39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99c0ca510d39467a8fd3a9ad77926902', 'request_id': 'req-e69783ca-9e78-4a0b-ae40-27bba7a9da39'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:12.749850 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072b690f-acf6-431b-8c29-425871e5c469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:12.750580 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072b690f-acf6-431b-8c29-425871e5c469 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:12.750867 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072b690f-acf6-431b-8c29-425871e5c469 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:12.751179 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072b690f-acf6-431b-8c29-425871e5c469 admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:12.751784 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072b690f-acf6-431b-8c29-425871e5c469 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:12.753381 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072b690f-acf6-431b-8c29-425871e5c469 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:12.765633 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1298/2587] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:46:12 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 07:46:12.769859 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072b690f-acf6-431b-8c29-425871e5c469 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:12.780434 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072b690f-acf6-431b-8c29-425871e5c469 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:12.785109 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4316ca6-021c-42dc-a55a-f1b77142447b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:12.785796 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4316ca6-021c-42dc-a55a-f1b77142447b admin admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:12.786069 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4316ca6-021c-42dc-a55a-f1b77142447b admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:12.786343 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4316ca6-021c-42dc-a55a-f1b77142447b admin admin] PATH_INFO: `/v3/projects/3dad55c07f754464bb200cddb6e5cbb3/users/644ccd81e7b04a54923bc868fca279ec/roles/053e12af84e943e699460a9d76c2072b` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:12.794639 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-e69783ca-9e78-4a0b-ae40-27bba7a9da39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99c0ca510d39467a8fd3a9ad77926902', 'request_id': 'req-e69783ca-9e78-4a0b-ae40-27bba7a9da39'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:12.802756 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4316ca6-021c-42dc-a55a-f1b77142447b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3dad55c07f754464bb200cddb6e5cbb3, user_id=644ccd81e7b04a54923bc868fca279ec, role_id=053e12af84e943e699460a9d76c2072b)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:12.808006 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4316ca6-021c-42dc-a55a-f1b77142447b admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:12.808198 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-e69783ca-9e78-4a0b-ae40-27bba7a9da39 admin admin] The token cache is being invalidate because project 99c0ca510d39467a8fd3a9ad77926902 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:46:12.808198 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-e69783ca-9e78-4a0b-ae40-27bba7a9da39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:12.809044 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-e69783ca-9e78-4a0b-ae40-27bba7a9da39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:12.811201 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1290/2588] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:46:12 2022] DELETE /identity/v3/projects/99c0ca510d39467a8fd3a9ad77926902 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 07:46:12.815888 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a193247b-0fcf-486f-a2b4-2087dfe3b0e4 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:12.816189 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a193247b-0fcf-486f-a2b4-2087dfe3b0e4 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:12.816460 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a193247b-0fcf-486f-a2b4-2087dfe3b0e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:12.831624 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:46:12.837485 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-a193247b-0fcf-486f-a2b4-2087dfe3b0e4 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:46:12.838762 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-a193247b-0fcf-486f-a2b4-2087dfe3b0e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:46:12.847704 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1291/2589] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:46:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 07:46:12.914875 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056947a5-2ab7-4eb3-8994-4248ac384ec2 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:12.915209 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056947a5-2ab7-4eb3-8994-4248ac384ec2 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:12.915430 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056947a5-2ab7-4eb3-8994-4248ac384ec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:12.916302 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:46:12.923121 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1299/2590] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:46:12 2022] PUT /identity/v3/projects/3dad55c07f754464bb200cddb6e5cbb3/users/644ccd81e7b04a54923bc868fca279ec/roles/053e12af84e943e699460a9d76c2072b => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 07:46:12.933535 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a437162-5ff6-4f2d-bdf6-4ec811e05f6b None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:12.933881 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a437162-5ff6-4f2d-bdf6-4ec811e05f6b None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:12.934158 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a437162-5ff6-4f2d-bdf6-4ec811e05f6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:12.937928 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-056947a5-2ab7-4eb3-8994-4248ac384ec2 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:46:12.943849 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-056947a5-2ab7-4eb3-8994-4248ac384ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:46:12.966349 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-0a437162-5ff6-4f2d-bdf6-4ec811e05f6b None None] MFA Rules not processed for user `644ccd81e7b04a54923bc868fca279ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:46:12.967669 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-0a437162-5ff6-4f2d-bdf6-4ec811e05f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:46:13.007954 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1292/2591] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:46:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 07:46:13.012249 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056947a5-2ab7-4eb3-8994-4248ac384ec2 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:13.021165 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056947a5-2ab7-4eb3-8994-4248ac384ec2 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:13.025737 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd448ba6-dd7a-4120-8e0d-dd6b597e3cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:13.026329 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd448ba6-dd7a-4120-8e0d-dd6b597e3cb5 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:13.026610 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd448ba6-dd7a-4120-8e0d-dd6b597e3cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:13.026925 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd448ba6-dd7a-4120-8e0d-dd6b597e3cb5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:13.027560 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd448ba6-dd7a-4120-8e0d-dd6b597e3cb5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:46:13.027859 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd448ba6-dd7a-4120-8e0d-dd6b597e3cb5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:13.030185 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd448ba6-dd7a-4120-8e0d-dd6b597e3cb5 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:13.037001 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1300/2592] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:46:12 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 07:46:13.041611 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a437162-5ff6-4f2d-bdf6-4ec811e05f6b None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:13.045777 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1293/2593] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 15 07:46:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:46:13.045916 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-0a437162-5ff6-4f2d-bdf6-4ec811e05f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:46:13.046922 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd448ba6-dd7a-4120-8e0d-dd6b597e3cb5 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:13.056327 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a437162-5ff6-4f2d-bdf6-4ec811e05f6b None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:13.056511 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd448ba6-dd7a-4120-8e0d-dd6b597e3cb5 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:13.061742 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbd5ea4-7c27-4581-bc23-ce1b6db6362e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:13.061892 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d5379b-0a46-4ce1-927a-a45c65b91075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:13.062274 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbd5ea4-7c27-4581-bc23-ce1b6db6362e admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:13.062450 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d5379b-0a46-4ce1-927a-a45c65b91075 admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:13.062594 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbd5ea4-7c27-4581-bc23-ce1b6db6362e admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:13.062730 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d5379b-0a46-4ce1-927a-a45c65b91075 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:13.062835 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbd5ea4-7c27-4581-bc23-ce1b6db6362e admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:13.063087 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d5379b-0a46-4ce1-927a-a45c65b91075 admin admin] PATH_INFO: `/v3/projects` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:13.063592 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbd5ea4-7c27-4581-bc23-ce1b6db6362e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:13.063703 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d5379b-0a46-4ce1-927a-a45c65b91075 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:13.065475 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbd5ea4-7c27-4581-bc23-ce1b6db6362e admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:13.065637 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d5379b-0a46-4ce1-927a-a45c65b91075 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:13.144395 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1301/2594] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:46:13 2022] POST /identity/v3/projects => generated 381 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 07:46:13.148475 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d5379b-0a46-4ce1-927a-a45c65b91075 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:13.159032 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1294/2595] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:46:13 2022] POST /identity/v3/projects => generated 365 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:46:13.160295 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbd5ea4-7c27-4581-bc23-ce1b6db6362e admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:13.211972 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d5379b-0a46-4ce1-927a-a45c65b91075 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:13.224230 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbd5ea4-7c27-4581-bc23-ce1b6db6362e admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:13.229073 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395cc698-6e59-4895-a0a2-a6e0b5bf240b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:13.229710 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395cc698-6e59-4895-a0a2-a6e0b5bf240b admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:13.229988 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395cc698-6e59-4895-a0a2-a6e0b5bf240b admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:13.230258 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395cc698-6e59-4895-a0a2-a6e0b5bf240b admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:13.230876 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395cc698-6e59-4895-a0a2-a6e0b5bf240b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:13.232919 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395cc698-6e59-4895-a0a2-a6e0b5bf240b admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:13.269905 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3037a82-81a3-4712-8b6b-f7373087e87e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:13.270516 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3037a82-81a3-4712-8b6b-f7373087e87e admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:13.270798 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3037a82-81a3-4712-8b6b-f7373087e87e admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:13.271075 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3037a82-81a3-4712-8b6b-f7373087e87e admin admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:13.271717 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3037a82-81a3-4712-8b6b-f7373087e87e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:13.273679 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3037a82-81a3-4712-8b6b-f7373087e87e admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:13.348810 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1295/2596] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:46:13 2022] POST /identity/v3/users => generated 283 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 07:46:13.352749 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395cc698-6e59-4895-a0a2-a6e0b5bf240b admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:13.363066 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395cc698-6e59-4895-a0a2-a6e0b5bf240b admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:13.367565 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aae6ae5-2382-40b5-86d9-a10012cef6ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:13.368129 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aae6ae5-2382-40b5-86d9-a10012cef6ef admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:13.368407 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aae6ae5-2382-40b5-86d9-a10012cef6ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:13.368676 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aae6ae5-2382-40b5-86d9-a10012cef6ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:13.369296 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aae6ae5-2382-40b5-86d9-a10012cef6ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:13.370927 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aae6ae5-2382-40b5-86d9-a10012cef6ef admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:13.377638 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1302/2597] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:46:13 2022] POST /identity/v3/users => generated 297 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:46:13.381621 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3037a82-81a3-4712-8b6b-f7373087e87e admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:13.388274 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1296/2598] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:46:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 07:46:13.388434 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:46:13.389299 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aae6ae5-2382-40b5-86d9-a10012cef6ef admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:13.392564 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3037a82-81a3-4712-8b6b-f7373087e87e admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:13.397481 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900e703f-a2b1-4a0c-a531-12355c6cce83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:13.398135 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900e703f-a2b1-4a0c-a531-12355c6cce83 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:13.398420 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900e703f-a2b1-4a0c-a531-12355c6cce83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:13.398697 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900e703f-a2b1-4a0c-a531-12355c6cce83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:13.398933 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aae6ae5-2382-40b5-86d9-a10012cef6ef admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:13.399339 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900e703f-a2b1-4a0c-a531-12355c6cce83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:13.400934 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900e703f-a2b1-4a0c-a531-12355c6cce83 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:13.404681 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf53f31-6a32-40a9-9b96-63dafa9fc5c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:13.405327 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf53f31-6a32-40a9-9b96-63dafa9fc5c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:13.405633 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf53f31-6a32-40a9-9b96-63dafa9fc5c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:13.405877 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf53f31-6a32-40a9-9b96-63dafa9fc5c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:13.406512 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf53f31-6a32-40a9-9b96-63dafa9fc5c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:13.408077 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf53f31-6a32-40a9-9b96-63dafa9fc5c7 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:13.412273 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1303/2599] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:46:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 07:46:13.416132 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900e703f-a2b1-4a0c-a531-12355c6cce83 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:13.420352 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1297/2600] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:46:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:46:13.424510 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf53f31-6a32-40a9-9b96-63dafa9fc5c7 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:13.427337 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900e703f-a2b1-4a0c-a531-12355c6cce83 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:13.432023 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc14369-3f4d-448e-85e2-d171a04421bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:13.432731 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc14369-3f4d-448e-85e2-d171a04421bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:13.432948 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc14369-3f4d-448e-85e2-d171a04421bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:13.433225 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc14369-3f4d-448e-85e2-d171a04421bc admin admin] PATH_INFO: `/v3/projects/82861236b039483a81e97c7ffe846892/users/cbd83cbc91534bd384d33226f2d9f829/roles/6bbb643e0d8d4bc0a9b27c26edf59216` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:13.435404 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf53f31-6a32-40a9-9b96-63dafa9fc5c7 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:13.440387 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d33017c-88c8-477a-ab7d-d9c41e30d860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:13.441054 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d33017c-88c8-477a-ab7d-d9c41e30d860 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:13.441330 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d33017c-88c8-477a-ab7d-d9c41e30d860 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:13.442028 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d33017c-88c8-477a-ab7d-d9c41e30d860 admin admin] PATH_INFO: `/v3/projects/e681e2f82201457eb7fb7e753807f155/users/52f0cd498d8c49c58d7289c55188f5f4/roles/053e12af84e943e699460a9d76c2072b` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:13.448016 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc14369-3f4d-448e-85e2-d171a04421bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=82861236b039483a81e97c7ffe846892, user_id=cbd83cbc91534bd384d33226f2d9f829, role_id=6bbb643e0d8d4bc0a9b27c26edf59216)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:13.451553 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc14369-3f4d-448e-85e2-d171a04421bc admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:13.457644 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d33017c-88c8-477a-ab7d-d9c41e30d860 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e681e2f82201457eb7fb7e753807f155, user_id=52f0cd498d8c49c58d7289c55188f5f4, role_id=053e12af84e943e699460a9d76c2072b)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:13.460269 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d33017c-88c8-477a-ab7d-d9c41e30d860 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:13.549435 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1304/2601] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:46:13 2022] PUT /identity/v3/projects/82861236b039483a81e97c7ffe846892/users/cbd83cbc91534bd384d33226f2d9f829/roles/6bbb643e0d8d4bc0a9b27c26edf59216 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 07:46:13.553243 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc14369-3f4d-448e-85e2-d171a04421bc admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:13.561551 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1298/2602] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:46:13 2022] PUT /identity/v3/projects/e681e2f82201457eb7fb7e753807f155/users/52f0cd498d8c49c58d7289c55188f5f4/roles/053e12af84e943e699460a9d76c2072b => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 07:46:13.582128 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b94b130-4d88-4570-a3cf-b4692b4bb6e8 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:13.582616 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b94b130-4d88-4570-a3cf-b4692b4bb6e8 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:13.582760 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b94b130-4d88-4570-a3cf-b4692b4bb6e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:13.603146 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:46:13.613539 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-7b94b130-4d88-4570-a3cf-b4692b4bb6e8 None None] MFA Rules not processed for user `52f0cd498d8c49c58d7289c55188f5f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:46:13.615059 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-7b94b130-4d88-4570-a3cf-b4692b4bb6e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:46:13.623412 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc14369-3f4d-448e-85e2-d171a04421bc admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:13.689567 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1299/2603] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:46:13 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:46:13.693554 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a246cb-e9a7-4f44-873b-cd205f5632c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:13.696496 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a246cb-e9a7-4f44-873b-cd205f5632c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:13.696843 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a246cb-e9a7-4f44-873b-cd205f5632c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:13.697124 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a246cb-e9a7-4f44-873b-cd205f5632c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:13.697821 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a246cb-e9a7-4f44-873b-cd205f5632c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:13.699434 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a246cb-e9a7-4f44-873b-cd205f5632c3 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:13.705313 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b94b130-4d88-4570-a3cf-b4692b4bb6e8 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:13.706854 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-7b94b130-4d88-4570-a3cf-b4692b4bb6e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:46:13.715810 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1305/2604] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:46:13 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 07:46:13.716821 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a246cb-e9a7-4f44-873b-cd205f5632c3 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:13.726450 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a246cb-e9a7-4f44-873b-cd205f5632c3 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:13.732000 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aa36b2-19b3-40b8-8f90-bfbe2b04c064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:13.732704 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aa36b2-19b3-40b8-8f90-bfbe2b04c064 admin admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:13.733088 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aa36b2-19b3-40b8-8f90-bfbe2b04c064 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:13.733248 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aa36b2-19b3-40b8-8f90-bfbe2b04c064 admin admin] PATH_INFO: `/v3/projects/82861236b039483a81e97c7ffe846892/users/cbd83cbc91534bd384d33226f2d9f829/roles/6bbb643e0d8d4bc0a9b27c26edf59216` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:13.735245 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aa36b2-19b3-40b8-8f90-bfbe2b04c064 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82861236b039483a81e97c7ffe846892, user_id=cbd83cbc91534bd384d33226f2d9f829, role_id=6bbb643e0d8d4bc0a9b27c26edf59216)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:13.737974 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aa36b2-19b3-40b8-8f90-bfbe2b04c064 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:13.749319 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1306/2605] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:46:13 2022] PUT /identity/v3/projects/82861236b039483a81e97c7ffe846892/users/cbd83cbc91534bd384d33226f2d9f829/roles/6bbb643e0d8d4bc0a9b27c26edf59216 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:46:13.759164 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11120115-a0f0-41a0-bddc-28d65e42788d None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:13.759451 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11120115-a0f0-41a0-bddc-28d65e42788d None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:13.759728 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11120115-a0f0-41a0-bddc-28d65e42788d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:13.769974 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b94b130-4d88-4570-a3cf-b4692b4bb6e8 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:13.790273 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-11120115-a0f0-41a0-bddc-28d65e42788d None None] MFA Rules not processed for user `cbd83cbc91534bd384d33226f2d9f829`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:46:13.791474 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-11120115-a0f0-41a0-bddc-28d65e42788d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:46:13.821941 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a82af9-8aca-45b2-89c5-ce24386b5cd6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecf6642b7b84d3c9756bb41ab3349d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:13.822550 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a82af9-8aca-45b2-89c5-ce24386b5cd6 service cinder] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:13.823115 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a82af9-8aca-45b2-89c5-ce24386b5cd6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:13.823115 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a82af9-8aca-45b2-89c5-ce24386b5cd6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:13.838913 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:46:13.869523 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1307/2606] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:46:13 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 07:46:13.878032 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11120115-a0f0-41a0-bddc-28d65e42788d None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:13.879404 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-11120115-a0f0-41a0-bddc-28d65e42788d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:46:13.891954 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a82af9-8aca-45b2-89c5-ce24386b5cd6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:13.892829 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11120115-a0f0-41a0-bddc-28d65e42788d None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:13.893555 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a82af9-8aca-45b2-89c5-ce24386b5cd6 service cinder] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:13.896497 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eedff18-73a8-49e0-aca9-a31c504becf5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecf6642b7b84d3c9756bb41ab3349d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:13.897093 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eedff18-73a8-49e0-aca9-a31c504becf5 service cinder] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:13.897379 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eedff18-73a8-49e0-aca9-a31c504becf5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:13.897801 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eedff18-73a8-49e0-aca9-a31c504becf5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:13.899592 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-3eedff18-73a8-49e0-aca9-a31c504becf5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:46:13.903069 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1300/2607] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:46:13 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 07:46:13.970207 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eedff18-73a8-49e0-aca9-a31c504becf5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:13.971734 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eedff18-73a8-49e0-aca9-a31c504becf5 service cinder] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:13.982439 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1308/2608] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:46:13 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 07:46:16.102949 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a82af9-8aca-45b2-89c5-ce24386b5cd6 service cinder] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:16.112095 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a82af9-8aca-45b2-89c5-ce24386b5cd6 service cinder] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:16.116225 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844ec559-29c6-499b-8298-015df329d700 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecf6642b7b84d3c9756bb41ab3349d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:16.116480 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844ec559-29c6-499b-8298-015df329d700 service cinder] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:16.116752 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844ec559-29c6-499b-8298-015df329d700 service cinder] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:16.117024 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844ec559-29c6-499b-8298-015df329d700 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:16.123264 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=100 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:46:16.177214 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844ec559-29c6-499b-8298-015df329d700 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:16.178828 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844ec559-29c6-499b-8298-015df329d700 service cinder] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:16.190362 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1301/2609] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:46:16 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:46:17.818226 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eedff18-73a8-49e0-aca9-a31c504becf5 service cinder] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:17.827436 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eedff18-73a8-49e0-aca9-a31c504becf5 service cinder] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:17.832150 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bd745d-5fb6-4ec3-afc2-19c8894b220c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecf6642b7b84d3c9756bb41ab3349d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:17.832753 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bd745d-5fb6-4ec3-afc2-19c8894b220c service cinder] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:17.833031 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bd745d-5fb6-4ec3-afc2-19c8894b220c service cinder] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:17.833300 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bd745d-5fb6-4ec3-afc2-19c8894b220c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:17.843824 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bd745d-5fb6-4ec3-afc2-19c8894b220c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:17.845377 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bd745d-5fb6-4ec3-afc2-19c8894b220c service cinder] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:17.857543 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1309/2610] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:46:17 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:46:22.785267 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844ec559-29c6-499b-8298-015df329d700 service cinder] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:22.860651 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844ec559-29c6-499b-8298-015df329d700 service cinder] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:22.865188 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb9ef7b-1a55-4dd0-bfba-e2952410f2ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:22.865855 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb9ef7b-1a55-4dd0-bfba-e2952410f2ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:22.866138 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb9ef7b-1a55-4dd0-bfba-e2952410f2ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:22.866408 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb9ef7b-1a55-4dd0-bfba-e2952410f2ba admin admin] PATH_INFO: `/v3/users/8db58b5e74574338ba8c16adc52e9c43` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:22.867521 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb9ef7b-1a55-4dd0-bfba-e2952410f2ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=8db58b5e74574338ba8c16adc52e9c43)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:22.869459 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb9ef7b-1a55-4dd0-bfba-e2952410f2ba admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:23.098261 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-bbb9ef7b-1a55-4dd0-bfba-e2952410f2ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8db58b5e74574338ba8c16adc52e9c43'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:23.098907 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-bbb9ef7b-1a55-4dd0-bfba-e2952410f2ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8db58b5e74574338ba8c16adc52e9c43'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:23.175409 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-bbb9ef7b-1a55-4dd0-bfba-e2952410f2ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8db58b5e74574338ba8c16adc52e9c43'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:23.176037 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-bbb9ef7b-1a55-4dd0-bfba-e2952410f2ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8db58b5e74574338ba8c16adc52e9c43'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:23.183692 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-bbb9ef7b-1a55-4dd0-bfba-e2952410f2ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8db58b5e74574338ba8c16adc52e9c43'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:23.202477 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1302/2611] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:46:22 2022] DELETE /identity/v3/users/8db58b5e74574338ba8c16adc52e9c43 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:46:23.207234 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e078645b-21b9-43d8-aaf8-cf41b356e347 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:23.207748 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e078645b-21b9-43d8-aaf8-cf41b356e347 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:23.207866 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e078645b-21b9-43d8-aaf8-cf41b356e347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:23.226646 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-e078645b-21b9-43d8-aaf8-cf41b356e347 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:46:23.228115 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-e078645b-21b9-43d8-aaf8-cf41b356e347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:46:23.285824 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1310/2612] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:46:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:46:23.295892 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb9ef7b-1a55-4dd0-bfba-e2952410f2ba admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:23.297277 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-bbb9ef7b-1a55-4dd0-bfba-e2952410f2ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:46:23.328728 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:46:23.354508 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb9ef7b-1a55-4dd0-bfba-e2952410f2ba admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:23.358719 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a53c21f-4117-492f-9b64-a8a8cf0d047c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:23.359432 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a53c21f-4117-492f-9b64-a8a8cf0d047c service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:23.359712 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a53c21f-4117-492f-9b64-a8a8cf0d047c service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:23.360023 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a53c21f-4117-492f-9b64-a8a8cf0d047c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:23.369194 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a53c21f-4117-492f-9b64-a8a8cf0d047c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:23.371046 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a53c21f-4117-492f-9b64-a8a8cf0d047c service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:23.381023 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1303/2613] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:46:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 07:46:23.771310 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e078645b-21b9-43d8-aaf8-cf41b356e347 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:23.772830 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-e078645b-21b9-43d8-aaf8-cf41b356e347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:46:23.786379 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e078645b-21b9-43d8-aaf8-cf41b356e347 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:23.791078 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47602364-367e-4a27-8487-7f36dc26784e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:23.791756 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47602364-367e-4a27-8487-7f36dc26784e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:23.792044 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47602364-367e-4a27-8487-7f36dc26784e admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:23.792321 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47602364-367e-4a27-8487-7f36dc26784e admin admin] PATH_INFO: `/v3/projects/87211e3d93d54012a968a182217df89f` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:23.793446 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47602364-367e-4a27-8487-7f36dc26784e admin admin] RBAC: Authorizing `identity:delete_project(project_id=87211e3d93d54012a968a182217df89f)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:23.796274 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47602364-367e-4a27-8487-7f36dc26784e admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:23.884539 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-47602364-367e-4a27-8487-7f36dc26784e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87211e3d93d54012a968a182217df89f', 'request_id': 'req-47602364-367e-4a27-8487-7f36dc26784e'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:23.891293 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-47602364-367e-4a27-8487-7f36dc26784e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87211e3d93d54012a968a182217df89f', 'request_id': 'req-47602364-367e-4a27-8487-7f36dc26784e'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:23.958021 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-47602364-367e-4a27-8487-7f36dc26784e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87211e3d93d54012a968a182217df89f', 'request_id': 'req-47602364-367e-4a27-8487-7f36dc26784e'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:23.960333 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:46:23.969833 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-47602364-367e-4a27-8487-7f36dc26784e admin admin] The token cache is being invalidate because project 87211e3d93d54012a968a182217df89f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:46:23.970124 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-47602364-367e-4a27-8487-7f36dc26784e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:23.970741 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-47602364-367e-4a27-8487-7f36dc26784e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:23.977063 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1311/2614] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:46:23 2022] DELETE /identity/v3/projects/87211e3d93d54012a968a182217df89f => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:46:23.977224 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a53c21f-4117-492f-9b64-a8a8cf0d047c service neutron] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:23.986179 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a53c21f-4117-492f-9b64-a8a8cf0d047c service neutron] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:24.043091 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bad136-123f-448c-ab0f-0e953d7ab7a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:24.043709 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bad136-123f-448c-ab0f-0e953d7ab7a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:24.043988 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bad136-123f-448c-ab0f-0e953d7ab7a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:24.044258 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bad136-123f-448c-ab0f-0e953d7ab7a2 admin admin] PATH_INFO: `/v3/users/644ccd81e7b04a54923bc868fca279ec` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:24.045361 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bad136-123f-448c-ab0f-0e953d7ab7a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=644ccd81e7b04a54923bc868fca279ec)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:24.047459 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bad136-123f-448c-ab0f-0e953d7ab7a2 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:24.263089 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-a3bad136-123f-448c-ab0f-0e953d7ab7a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '644ccd81e7b04a54923bc868fca279ec'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:24.263791 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-a3bad136-123f-448c-ab0f-0e953d7ab7a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '644ccd81e7b04a54923bc868fca279ec'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:24.341613 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-a3bad136-123f-448c-ab0f-0e953d7ab7a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '644ccd81e7b04a54923bc868fca279ec'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:24.342227 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-a3bad136-123f-448c-ab0f-0e953d7ab7a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '644ccd81e7b04a54923bc868fca279ec'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:24.349685 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-a3bad136-123f-448c-ab0f-0e953d7ab7a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '644ccd81e7b04a54923bc868fca279ec'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:24.368355 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1304/2615] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:46:23 2022] DELETE /identity/v3/users/644ccd81e7b04a54923bc868fca279ec => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:46:24.777909 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47602364-367e-4a27-8487-7f36dc26784e admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:24.780480 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-47602364-367e-4a27-8487-7f36dc26784e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:46:24.796264 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47602364-367e-4a27-8487-7f36dc26784e admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:24.864040 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72de45aa-bb57-4d9e-a121-d2e3af6c1664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:24.864677 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72de45aa-bb57-4d9e-a121-d2e3af6c1664 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:24.864982 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72de45aa-bb57-4d9e-a121-d2e3af6c1664 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:24.865256 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72de45aa-bb57-4d9e-a121-d2e3af6c1664 admin admin] PATH_INFO: `/v3/projects/3dad55c07f754464bb200cddb6e5cbb3` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:24.866407 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72de45aa-bb57-4d9e-a121-d2e3af6c1664 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3dad55c07f754464bb200cddb6e5cbb3)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:24.868310 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72de45aa-bb57-4d9e-a121-d2e3af6c1664 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:24.976140 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-72de45aa-bb57-4d9e-a121-d2e3af6c1664 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3dad55c07f754464bb200cddb6e5cbb3', 'request_id': 'req-72de45aa-bb57-4d9e-a121-d2e3af6c1664'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:24.983294 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-72de45aa-bb57-4d9e-a121-d2e3af6c1664 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3dad55c07f754464bb200cddb6e5cbb3', 'request_id': 'req-72de45aa-bb57-4d9e-a121-d2e3af6c1664'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:25.049094 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-72de45aa-bb57-4d9e-a121-d2e3af6c1664 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3dad55c07f754464bb200cddb6e5cbb3', 'request_id': 'req-72de45aa-bb57-4d9e-a121-d2e3af6c1664'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:25.060685 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-72de45aa-bb57-4d9e-a121-d2e3af6c1664 admin admin] The token cache is being invalidate because project 3dad55c07f754464bb200cddb6e5cbb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:46:25.060982 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-72de45aa-bb57-4d9e-a121-d2e3af6c1664 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:25.061650 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-72de45aa-bb57-4d9e-a121-d2e3af6c1664 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:25.063145 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1312/2616] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:46:24 2022] DELETE /identity/v3/projects/3dad55c07f754464bb200cddb6e5cbb3 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:46:25.067314 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bad136-123f-448c-ab0f-0e953d7ab7a2 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:25.068766 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-a3bad136-123f-448c-ab0f-0e953d7ab7a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:46:25.082708 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bad136-123f-448c-ab0f-0e953d7ab7a2 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:25.129224 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:46:25.141731 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b9bd12-c80e-497b-ae4a-542ec32da00c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:25.142303 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b9bd12-c80e-497b-ae4a-542ec32da00c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:25.142589 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b9bd12-c80e-497b-ae4a-542ec32da00c admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:25.142863 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b9bd12-c80e-497b-ae4a-542ec32da00c admin admin] PATH_INFO: `/v3/users/cbd83cbc91534bd384d33226f2d9f829` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:25.143987 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b9bd12-c80e-497b-ae4a-542ec32da00c admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbd83cbc91534bd384d33226f2d9f829)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:25.145979 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b9bd12-c80e-497b-ae4a-542ec32da00c admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:25.296802 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-a4b9bd12-c80e-497b-ae4a-542ec32da00c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbd83cbc91534bd384d33226f2d9f829'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:25.297459 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-a4b9bd12-c80e-497b-ae4a-542ec32da00c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbd83cbc91534bd384d33226f2d9f829'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:25.369371 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-a4b9bd12-c80e-497b-ae4a-542ec32da00c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbd83cbc91534bd384d33226f2d9f829'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:25.370034 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-a4b9bd12-c80e-497b-ae4a-542ec32da00c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbd83cbc91534bd384d33226f2d9f829'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:25.378139 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-a4b9bd12-c80e-497b-ae4a-542ec32da00c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbd83cbc91534bd384d33226f2d9f829'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:25.395373 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1305/2617] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:46:25 2022] DELETE /identity/v3/users/cbd83cbc91534bd384d33226f2d9f829 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 07:46:25.748373 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72de45aa-bb57-4d9e-a121-d2e3af6c1664 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:25.749835 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-72de45aa-bb57-4d9e-a121-d2e3af6c1664 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:46:25.764453 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72de45aa-bb57-4d9e-a121-d2e3af6c1664 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:25.825500 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed6312f-4ca1-43b8-9e85-bffca9e39e2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:25.826197 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed6312f-4ca1-43b8-9e85-bffca9e39e2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:25.826448 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed6312f-4ca1-43b8-9e85-bffca9e39e2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:25.826722 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed6312f-4ca1-43b8-9e85-bffca9e39e2d admin admin] PATH_INFO: `/v3/projects/82861236b039483a81e97c7ffe846892` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:25.827838 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed6312f-4ca1-43b8-9e85-bffca9e39e2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=82861236b039483a81e97c7ffe846892)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:25.829826 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed6312f-4ca1-43b8-9e85-bffca9e39e2d admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:25.854194 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:46:25.967462 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-1ed6312f-4ca1-43b8-9e85-bffca9e39e2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82861236b039483a81e97c7ffe846892', 'request_id': 'req-1ed6312f-4ca1-43b8-9e85-bffca9e39e2d'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:25.974997 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-1ed6312f-4ca1-43b8-9e85-bffca9e39e2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82861236b039483a81e97c7ffe846892', 'request_id': 'req-1ed6312f-4ca1-43b8-9e85-bffca9e39e2d'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:26.048889 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-1ed6312f-4ca1-43b8-9e85-bffca9e39e2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82861236b039483a81e97c7ffe846892', 'request_id': 'req-1ed6312f-4ca1-43b8-9e85-bffca9e39e2d'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:26.059983 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-1ed6312f-4ca1-43b8-9e85-bffca9e39e2d admin admin] The token cache is being invalidate because project 82861236b039483a81e97c7ffe846892 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:46:26.060322 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-1ed6312f-4ca1-43b8-9e85-bffca9e39e2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:26.061080 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-1ed6312f-4ca1-43b8-9e85-bffca9e39e2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:26.062819 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1313/2618] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:46:25 2022] DELETE /identity/v3/projects/82861236b039483a81e97c7ffe846892 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:46:26.067641 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ad7af7-2227-43c3-9977-b97abb05fcdd None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:26.068099 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ad7af7-2227-43c3-9977-b97abb05fcdd None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:26.068190 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ad7af7-2227-43c3-9977-b97abb05fcdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:26.088325 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-d9ad7af7-2227-43c3-9977-b97abb05fcdd None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:46:26.089523 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-d9ad7af7-2227-43c3-9977-b97abb05fcdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:46:26.148826 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1306/2619] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:46:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:46:26.158240 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee09b18e-3871-42dc-9567-b5e9d5af16a6 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:26.158519 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee09b18e-3871-42dc-9567-b5e9d5af16a6 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:26.158791 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee09b18e-3871-42dc-9567-b5e9d5af16a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:26.178008 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-ee09b18e-3871-42dc-9567-b5e9d5af16a6 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:46:26.179188 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-ee09b18e-3871-42dc-9567-b5e9d5af16a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:46:26.187377 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1314/2620] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:46:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:46:26.191468 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ad7af7-2227-43c3-9977-b97abb05fcdd None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:26.201691 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ad7af7-2227-43c3-9977-b97abb05fcdd None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:26.206316 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0613cdc8-6b9b-4066-b746-3828eb36f44c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:26.206914 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0613cdc8-6b9b-4066-b746-3828eb36f44c admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:26.207192 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0613cdc8-6b9b-4066-b746-3828eb36f44c admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:26.207460 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0613cdc8-6b9b-4066-b746-3828eb36f44c admin admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:26.208100 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0613cdc8-6b9b-4066-b746-3828eb36f44c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:46:26.208400 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0613cdc8-6b9b-4066-b746-3828eb36f44c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:26.210050 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0613cdc8-6b9b-4066-b746-3828eb36f44c admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:26.222229 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1307/2621] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 15 07:46:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:46:26.226456 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee09b18e-3871-42dc-9567-b5e9d5af16a6 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:26.235827 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee09b18e-3871-42dc-9567-b5e9d5af16a6 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:26.240409 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734cebff-55c2-4141-892f-52f7d62ebc53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:26.241023 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734cebff-55c2-4141-892f-52f7d62ebc53 admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:26.241299 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734cebff-55c2-4141-892f-52f7d62ebc53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:26.241568 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734cebff-55c2-4141-892f-52f7d62ebc53 admin admin] PATH_INFO: `/v3/projects` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:26.242224 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734cebff-55c2-4141-892f-52f7d62ebc53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:26.244092 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734cebff-55c2-4141-892f-52f7d62ebc53 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:26.369709 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1315/2622] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:46:26 2022] POST /identity/v3/projects => generated 385 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:46:26.374458 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0613cdc8-6b9b-4066-b746-3828eb36f44c admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:26.384450 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0613cdc8-6b9b-4066-b746-3828eb36f44c admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:26.436717 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:46:26.443614 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f41b1a-7887-4831-84c3-dc03d27892f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:26.444185 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f41b1a-7887-4831-84c3-dc03d27892f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:26.444469 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f41b1a-7887-4831-84c3-dc03d27892f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:26.444744 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f41b1a-7887-4831-84c3-dc03d27892f2 admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:26.445381 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f41b1a-7887-4831-84c3-dc03d27892f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:26.447452 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f41b1a-7887-4831-84c3-dc03d27892f2 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:26.524199 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1308/2623] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:46:26 2022] POST /identity/v3/users => generated 293 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 07:46:26.528536 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734cebff-55c2-4141-892f-52f7d62ebc53 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:26.538986 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734cebff-55c2-4141-892f-52f7d62ebc53 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:26.543739 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc94c5d-05f5-4ab9-b73e-a65ad8160de3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:26.544358 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc94c5d-05f5-4ab9-b73e-a65ad8160de3 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:26.544673 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc94c5d-05f5-4ab9-b73e-a65ad8160de3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:26.545039 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc94c5d-05f5-4ab9-b73e-a65ad8160de3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:26.545818 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc94c5d-05f5-4ab9-b73e-a65ad8160de3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:26.547414 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc94c5d-05f5-4ab9-b73e-a65ad8160de3 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:26.557619 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1316/2624] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:46:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:46:26.562193 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f41b1a-7887-4831-84c3-dc03d27892f2 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:26.572073 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f41b1a-7887-4831-84c3-dc03d27892f2 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:26.578034 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4af7177-43d4-470e-adc5-ed70f4296753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:26.578669 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4af7177-43d4-470e-adc5-ed70f4296753 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:26.579000 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4af7177-43d4-470e-adc5-ed70f4296753 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:26.579245 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4af7177-43d4-470e-adc5-ed70f4296753 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:26.579889 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4af7177-43d4-470e-adc5-ed70f4296753 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:26.581470 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4af7177-43d4-470e-adc5-ed70f4296753 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:26.592444 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1309/2625] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:46:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 07:46:26.596691 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc94c5d-05f5-4ab9-b73e-a65ad8160de3 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:26.607008 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc94c5d-05f5-4ab9-b73e-a65ad8160de3 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:26.611692 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec60be0d-5cc0-4bd0-9f64-056185cf891a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:26.612332 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec60be0d-5cc0-4bd0-9f64-056185cf891a admin admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:26.612608 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec60be0d-5cc0-4bd0-9f64-056185cf891a admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:26.612878 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec60be0d-5cc0-4bd0-9f64-056185cf891a admin admin] PATH_INFO: `/v3/projects/165d45f48d9d4f66ab9f3af1b64ff778/users/181cdc1ff2cf4fa4a4e84178cc91b6a7/roles/053e12af84e943e699460a9d76c2072b` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:26.628022 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec60be0d-5cc0-4bd0-9f64-056185cf891a admin admin] RBAC: Authorizing `identity:create_grant(project_id=165d45f48d9d4f66ab9f3af1b64ff778, user_id=181cdc1ff2cf4fa4a4e84178cc91b6a7, role_id=053e12af84e943e699460a9d76c2072b)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:26.630705 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec60be0d-5cc0-4bd0-9f64-056185cf891a admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:26.712031 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1317/2626] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:46:26 2022] PUT /identity/v3/projects/165d45f48d9d4f66ab9f3af1b64ff778/users/181cdc1ff2cf4fa4a4e84178cc91b6a7/roles/053e12af84e943e699460a9d76c2072b => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:46:26.722450 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46495c6c-b591-4f51-adb7-924bf2b2be1d None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:26.722780 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46495c6c-b591-4f51-adb7-924bf2b2be1d None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:26.723096 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46495c6c-b591-4f51-adb7-924bf2b2be1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:26.754016 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-46495c6c-b591-4f51-adb7-924bf2b2be1d None None] MFA Rules not processed for user `181cdc1ff2cf4fa4a4e84178cc91b6a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:46:26.755265 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-46495c6c-b591-4f51-adb7-924bf2b2be1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:46:26.819679 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1310/2627] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:46:26 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:46:26.824225 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec60be0d-5cc0-4bd0-9f64-056185cf891a admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:26.890618 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec60be0d-5cc0-4bd0-9f64-056185cf891a admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:26.895289 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c812d151-99a4-4472-bd6e-35b436dcbb51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:26.895895 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c812d151-99a4-4472-bd6e-35b436dcbb51 admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:26.896175 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c812d151-99a4-4472-bd6e-35b436dcbb51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:26.896449 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c812d151-99a4-4472-bd6e-35b436dcbb51 admin admin] PATH_INFO: `/v3/projects` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:26.897086 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c812d151-99a4-4472-bd6e-35b436dcbb51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:26.899028 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c812d151-99a4-4472-bd6e-35b436dcbb51 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:27.006039 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1318/2628] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:46:26 2022] POST /identity/v3/projects => generated 385 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:46:27.010357 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46495c6c-b591-4f51-adb7-924bf2b2be1d None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:27.021175 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46495c6c-b591-4f51-adb7-924bf2b2be1d None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:27.079826 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e5fa2d-9290-4014-87a3-47839ba24249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:27.080445 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e5fa2d-9290-4014-87a3-47839ba24249 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:27.080706 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e5fa2d-9290-4014-87a3-47839ba24249 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:27.080985 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e5fa2d-9290-4014-87a3-47839ba24249 admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:27.081708 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e5fa2d-9290-4014-87a3-47839ba24249 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:27.083767 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e5fa2d-9290-4014-87a3-47839ba24249 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:27.170319 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1311/2629] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:46:27 2022] POST /identity/v3/users => generated 293 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:46:27.174352 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c812d151-99a4-4472-bd6e-35b436dcbb51 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:27.184662 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c812d151-99a4-4472-bd6e-35b436dcbb51 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:27.189336 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d63b43f-8946-4fc2-8a36-be90b57b95f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:27.191422 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d63b43f-8946-4fc2-8a36-be90b57b95f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:27.191422 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d63b43f-8946-4fc2-8a36-be90b57b95f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:27.191422 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d63b43f-8946-4fc2-8a36-be90b57b95f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:27.191422 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d63b43f-8946-4fc2-8a36-be90b57b95f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:27.192785 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d63b43f-8946-4fc2-8a36-be90b57b95f7 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:27.203900 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1319/2630] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:46:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:46:27.207788 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e5fa2d-9290-4014-87a3-47839ba24249 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:27.218619 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e5fa2d-9290-4014-87a3-47839ba24249 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:27.223362 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813456c4-1497-41ae-a7b4-c998e8d59d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:27.223964 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813456c4-1497-41ae-a7b4-c998e8d59d51 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:27.224243 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813456c4-1497-41ae-a7b4-c998e8d59d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:27.224519 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813456c4-1497-41ae-a7b4-c998e8d59d51 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:27.225177 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813456c4-1497-41ae-a7b4-c998e8d59d51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:27.226784 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813456c4-1497-41ae-a7b4-c998e8d59d51 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:27.237592 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1312/2631] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:46:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:46:27.241730 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d63b43f-8946-4fc2-8a36-be90b57b95f7 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:27.252753 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d63b43f-8946-4fc2-8a36-be90b57b95f7 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:27.258070 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa41c4b-9ef2-48cf-9fd3-c3d555648f4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:27.258730 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa41c4b-9ef2-48cf-9fd3-c3d555648f4a admin admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:27.259015 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa41c4b-9ef2-48cf-9fd3-c3d555648f4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:27.259455 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa41c4b-9ef2-48cf-9fd3-c3d555648f4a admin admin] PATH_INFO: `/v3/projects/86d78bc517bc44f186c0eb1779a22dd6/users/b6a8e1733f7e4d7b8f9c03389905c102/roles/053e12af84e943e699460a9d76c2072b` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:27.275411 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa41c4b-9ef2-48cf-9fd3-c3d555648f4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=86d78bc517bc44f186c0eb1779a22dd6, user_id=b6a8e1733f7e4d7b8f9c03389905c102, role_id=053e12af84e943e699460a9d76c2072b)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:27.278238 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa41c4b-9ef2-48cf-9fd3-c3d555648f4a admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:27.358368 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=89,keystone:DELETE=6,keystone:INSERT=5 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:46:27.363965 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1320/2632] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:46:27 2022] PUT /identity/v3/projects/86d78bc517bc44f186c0eb1779a22dd6/users/b6a8e1733f7e4d7b8f9c03389905c102/roles/053e12af84e943e699460a9d76c2072b => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:46:27.383405 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e89f617-0562-4b1b-8f59-0b9caa61e637 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:27.383698 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e89f617-0562-4b1b-8f59-0b9caa61e637 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:27.384006 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e89f617-0562-4b1b-8f59-0b9caa61e637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:27.414354 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-2e89f617-0562-4b1b-8f59-0b9caa61e637 None None] MFA Rules not processed for user `b6a8e1733f7e4d7b8f9c03389905c102`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:46:27.415614 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-2e89f617-0562-4b1b-8f59-0b9caa61e637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:46:27.454965 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:46:27.490320 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1313/2633] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:46:27 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:46:27.494953 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa41c4b-9ef2-48cf-9fd3-c3d555648f4a admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:27.561826 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa41c4b-9ef2-48cf-9fd3-c3d555648f4a admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:27.566737 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e0287a-ee6b-4dc0-8ce0-3bbbec855596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:27.567371 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e0287a-ee6b-4dc0-8ce0-3bbbec855596 admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:27.567689 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e0287a-ee6b-4dc0-8ce0-3bbbec855596 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:27.567966 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e0287a-ee6b-4dc0-8ce0-3bbbec855596 admin admin] PATH_INFO: `/v3/projects` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:27.568613 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e0287a-ee6b-4dc0-8ce0-3bbbec855596 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:27.570624 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e0287a-ee6b-4dc0-8ce0-3bbbec855596 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:27.731011 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1321/2634] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:46:27 2022] POST /identity/v3/projects => generated 385 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:46:27.735474 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e89f617-0562-4b1b-8f59-0b9caa61e637 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:27.749864 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e89f617-0562-4b1b-8f59-0b9caa61e637 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:27.802836 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea95aa1-9033-41d1-bf76-d30d315c1578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:27.803440 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea95aa1-9033-41d1-bf76-d30d315c1578 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:27.803717 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea95aa1-9033-41d1-bf76-d30d315c1578 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:27.803987 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea95aa1-9033-41d1-bf76-d30d315c1578 admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:27.804623 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea95aa1-9033-41d1-bf76-d30d315c1578 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:27.806544 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea95aa1-9033-41d1-bf76-d30d315c1578 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:27.906883 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1314/2635] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:46:27 2022] POST /identity/v3/users => generated 299 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:46:27.910907 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e0287a-ee6b-4dc0-8ce0-3bbbec855596 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:27.922120 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e0287a-ee6b-4dc0-8ce0-3bbbec855596 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:27.927078 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac656935-4bf1-4f62-b480-e1a3d8b64e15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:27.927795 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac656935-4bf1-4f62-b480-e1a3d8b64e15 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:27.928119 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac656935-4bf1-4f62-b480-e1a3d8b64e15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:27.928398 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac656935-4bf1-4f62-b480-e1a3d8b64e15 admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:27.929170 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac656935-4bf1-4f62-b480-e1a3d8b64e15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:27.930805 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac656935-4bf1-4f62-b480-e1a3d8b64e15 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:27.946002 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1322/2636] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:46:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:46:27.947851 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea95aa1-9033-41d1-bf76-d30d315c1578 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:27.958516 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea95aa1-9033-41d1-bf76-d30d315c1578 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:27.963365 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5940893b-a2f9-4070-8003-3184634ed4d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:27.964016 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5940893b-a2f9-4070-8003-3184634ed4d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:27.964323 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5940893b-a2f9-4070-8003-3184634ed4d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:27.964617 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5940893b-a2f9-4070-8003-3184634ed4d5 admin admin] PATH_INFO: `/v3/projects/9114fb76157a47e78ae03e889a9db861/users/8273028285fe4cb4ae5f89a113765a7a/roles/6bbb643e0d8d4bc0a9b27c26edf59216` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:27.980961 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5940893b-a2f9-4070-8003-3184634ed4d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9114fb76157a47e78ae03e889a9db861, user_id=8273028285fe4cb4ae5f89a113765a7a, role_id=6bbb643e0d8d4bc0a9b27c26edf59216)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:27.984539 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5940893b-a2f9-4070-8003-3184634ed4d5 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:28.077459 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1315/2637] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:46:27 2022] PUT /identity/v3/projects/9114fb76157a47e78ae03e889a9db861/users/8273028285fe4cb4ae5f89a113765a7a/roles/6bbb643e0d8d4bc0a9b27c26edf59216 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:46:28.083920 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac656935-4bf1-4f62-b480-e1a3d8b64e15 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:28.094804 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac656935-4bf1-4f62-b480-e1a3d8b64e15 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:28.155208 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c39ae8-98a5-4b09-871c-368433c772f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:28.155825 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c39ae8-98a5-4b09-871c-368433c772f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:28.156156 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c39ae8-98a5-4b09-871c-368433c772f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:28.156417 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c39ae8-98a5-4b09-871c-368433c772f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:28.157070 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c39ae8-98a5-4b09-871c-368433c772f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:28.158738 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c39ae8-98a5-4b09-871c-368433c772f8 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:28.168786 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1323/2638] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:46:28 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:46:28.173618 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5940893b-a2f9-4070-8003-3184634ed4d5 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:28.184035 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5940893b-a2f9-4070-8003-3184634ed4d5 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:28.189268 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18390938-ac97-415b-a1b3-c74190a5a351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:28.195105 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18390938-ac97-415b-a1b3-c74190a5a351 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:28.195105 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18390938-ac97-415b-a1b3-c74190a5a351 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:28.195105 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18390938-ac97-415b-a1b3-c74190a5a351 admin admin] PATH_INFO: `/v3/projects/9114fb76157a47e78ae03e889a9db861/users/8273028285fe4cb4ae5f89a113765a7a/roles/6bbb643e0d8d4bc0a9b27c26edf59216` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:28.195105 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18390938-ac97-415b-a1b3-c74190a5a351 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9114fb76157a47e78ae03e889a9db861, user_id=8273028285fe4cb4ae5f89a113765a7a, role_id=6bbb643e0d8d4bc0a9b27c26edf59216)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:28.195105 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18390938-ac97-415b-a1b3-c74190a5a351 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:28.205549 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1316/2639] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:46:28 2022] PUT /identity/v3/projects/9114fb76157a47e78ae03e889a9db861/users/8273028285fe4cb4ae5f89a113765a7a/roles/6bbb643e0d8d4bc0a9b27c26edf59216 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:46:28.217782 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704debf5-3861-471c-9e0b-e3c8fc6f48a3 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:28.218106 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704debf5-3861-471c-9e0b-e3c8fc6f48a3 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:28.218388 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704debf5-3861-471c-9e0b-e3c8fc6f48a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:28.253128 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-704debf5-3861-471c-9e0b-e3c8fc6f48a3 None None] MFA Rules not processed for user `8273028285fe4cb4ae5f89a113765a7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:46:28.254548 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-704debf5-3861-471c-9e0b-e3c8fc6f48a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:46:28.323791 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:46:28.335003 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1324/2640] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:46:28 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:46:28.346951 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18390938-ac97-415b-a1b3-c74190a5a351 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:28.348628 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-18390938-ac97-415b-a1b3-c74190a5a351 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:46:28.406522 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18390938-ac97-415b-a1b3-c74190a5a351 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:28.410337 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7ddf2a-801e-4312-8f1b-4581bb89ca02 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecf6642b7b84d3c9756bb41ab3349d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:28.410950 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7ddf2a-801e-4312-8f1b-4581bb89ca02 service cinder] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:28.411234 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7ddf2a-801e-4312-8f1b-4581bb89ca02 service cinder] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:28.411509 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7ddf2a-801e-4312-8f1b-4581bb89ca02 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:28.449945 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:46:28.467489 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7ddf2a-801e-4312-8f1b-4581bb89ca02 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:28.468900 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7ddf2a-801e-4312-8f1b-4581bb89ca02 service cinder] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:28.477995 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1317/2641] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:46:28 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:46:30.391476 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704debf5-3861-471c-9e0b-e3c8fc6f48a3 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:30.392888 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-704debf5-3861-471c-9e0b-e3c8fc6f48a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:46:30.404763 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704debf5-3861-471c-9e0b-e3c8fc6f48a3 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:30.408316 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93094167-2cf5-4a5f-b410-6db0652e6910 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecf6642b7b84d3c9756bb41ab3349d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:30.408915 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93094167-2cf5-4a5f-b410-6db0652e6910 service cinder] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:30.409195 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93094167-2cf5-4a5f-b410-6db0652e6910 service cinder] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:30.409464 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93094167-2cf5-4a5f-b410-6db0652e6910 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:30.467323 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93094167-2cf5-4a5f-b410-6db0652e6910 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:30.468765 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93094167-2cf5-4a5f-b410-6db0652e6910 service cinder] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:30.477778 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1325/2642] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:46:30 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:46:31.936045 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7ddf2a-801e-4312-8f1b-4581bb89ca02 service cinder] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:31.955118 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7ddf2a-801e-4312-8f1b-4581bb89ca02 service cinder] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:31.958894 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b095f7de-8ed0-4e69-89c4-79a04f341db8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecf6642b7b84d3c9756bb41ab3349d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:31.959580 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b095f7de-8ed0-4e69-89c4-79a04f341db8 service cinder] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:31.959768 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b095f7de-8ed0-4e69-89c4-79a04f341db8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:31.960096 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b095f7de-8ed0-4e69-89c4-79a04f341db8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:31.970730 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b095f7de-8ed0-4e69-89c4-79a04f341db8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:31.972236 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b095f7de-8ed0-4e69-89c4-79a04f341db8 service cinder] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:31.982846 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1318/2643] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:46:31 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:46:36.818138 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93094167-2cf5-4a5f-b410-6db0652e6910 service cinder] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:36.894086 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93094167-2cf5-4a5f-b410-6db0652e6910 service cinder] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:36.898597 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8cd838-ff82-4ea2-9a69-b207585baf65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:36.899237 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8cd838-ff82-4ea2-9a69-b207585baf65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:36.899516 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8cd838-ff82-4ea2-9a69-b207585baf65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:36.899789 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8cd838-ff82-4ea2-9a69-b207585baf65 admin admin] PATH_INFO: `/v3/users/181cdc1ff2cf4fa4a4e84178cc91b6a7` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:36.900908 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8cd838-ff82-4ea2-9a69-b207585baf65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=181cdc1ff2cf4fa4a4e84178cc91b6a7)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:36.902924 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8cd838-ff82-4ea2-9a69-b207585baf65 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:37.072826 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-6e8cd838-ff82-4ea2-9a69-b207585baf65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '181cdc1ff2cf4fa4a4e84178cc91b6a7'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:37.137857 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-6e8cd838-ff82-4ea2-9a69-b207585baf65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '181cdc1ff2cf4fa4a4e84178cc91b6a7'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:37.138491 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-6e8cd838-ff82-4ea2-9a69-b207585baf65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '181cdc1ff2cf4fa4a4e84178cc91b6a7'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:37.139099 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-6e8cd838-ff82-4ea2-9a69-b207585baf65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '181cdc1ff2cf4fa4a4e84178cc91b6a7'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:37.146989 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-6e8cd838-ff82-4ea2-9a69-b207585baf65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '181cdc1ff2cf4fa4a4e84178cc91b6a7'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:37.166262 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1326/2644] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:46:36 2022] DELETE /identity/v3/users/181cdc1ff2cf4fa4a4e84178cc91b6a7 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:46:37.171522 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf2c65b-e4dc-43b6-94f1-4ae6f771419f None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:37.174974 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf2c65b-e4dc-43b6-94f1-4ae6f771419f None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:37.174974 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf2c65b-e4dc-43b6-94f1-4ae6f771419f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:37.232524 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-dcf2c65b-e4dc-43b6-94f1-4ae6f771419f None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:46:37.233754 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-dcf2c65b-e4dc-43b6-94f1-4ae6f771419f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:46:37.332427 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1319/2645] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:46:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:46:37.343770 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8cd838-ff82-4ea2-9a69-b207585baf65 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:37.345419 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-6e8cd838-ff82-4ea2-9a69-b207585baf65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:46:37.361546 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:46:37.427314 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8cd838-ff82-4ea2-9a69-b207585baf65 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:37.431447 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3653d10-1c8b-42f5-a174-041be883f4e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:37.434832 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3653d10-1c8b-42f5-a174-041be883f4e0 service neutron] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:37.434832 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3653d10-1c8b-42f5-a174-041be883f4e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:37.434832 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3653d10-1c8b-42f5-a174-041be883f4e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:37.444419 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3653d10-1c8b-42f5-a174-041be883f4e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:37.445904 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3653d10-1c8b-42f5-a174-041be883f4e0 service neutron] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:37.456530 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1327/2646] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:46:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 07:46:38.089242 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf2c65b-e4dc-43b6-94f1-4ae6f771419f None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:38.090004 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-dcf2c65b-e4dc-43b6-94f1-4ae6f771419f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:46:38.103604 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf2c65b-e4dc-43b6-94f1-4ae6f771419f None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:38.108988 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a1e433-4798-42c6-9c72-754a223890c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:38.113051 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a1e433-4798-42c6-9c72-754a223890c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:38.113051 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a1e433-4798-42c6-9c72-754a223890c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:38.113051 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a1e433-4798-42c6-9c72-754a223890c0 admin admin] PATH_INFO: `/v3/projects/165d45f48d9d4f66ab9f3af1b64ff778` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:38.113051 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a1e433-4798-42c6-9c72-754a223890c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=165d45f48d9d4f66ab9f3af1b64ff778)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:38.113726 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a1e433-4798-42c6-9c72-754a223890c0 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:38.221181 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-e0a1e433-4798-42c6-9c72-754a223890c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '165d45f48d9d4f66ab9f3af1b64ff778', 'request_id': 'req-e0a1e433-4798-42c6-9c72-754a223890c0'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:38.224067 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:46:38.229017 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-e0a1e433-4798-42c6-9c72-754a223890c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '165d45f48d9d4f66ab9f3af1b64ff778', 'request_id': 'req-e0a1e433-4798-42c6-9c72-754a223890c0'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:38.304433 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-e0a1e433-4798-42c6-9c72-754a223890c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '165d45f48d9d4f66ab9f3af1b64ff778', 'request_id': 'req-e0a1e433-4798-42c6-9c72-754a223890c0'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:38.318421 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-e0a1e433-4798-42c6-9c72-754a223890c0 admin admin] The token cache is being invalidate because project 165d45f48d9d4f66ab9f3af1b64ff778 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:46:38.318999 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-e0a1e433-4798-42c6-9c72-754a223890c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:38.319658 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-e0a1e433-4798-42c6-9c72-754a223890c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:38.326058 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1320/2647] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:46:38 2022] DELETE /identity/v3/projects/165d45f48d9d4f66ab9f3af1b64ff778 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:46:38.327697 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3653d10-1c8b-42f5-a174-041be883f4e0 service neutron] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:38.339591 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3653d10-1c8b-42f5-a174-041be883f4e0 service neutron] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:38.405545 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f104ef-92b4-4a5c-b379-4a30f4a4b2e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:38.406127 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f104ef-92b4-4a5c-b379-4a30f4a4b2e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:38.406410 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f104ef-92b4-4a5c-b379-4a30f4a4b2e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:38.406683 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f104ef-92b4-4a5c-b379-4a30f4a4b2e6 admin admin] PATH_INFO: `/v3/users/b6a8e1733f7e4d7b8f9c03389905c102` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:38.407871 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f104ef-92b4-4a5c-b379-4a30f4a4b2e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6a8e1733f7e4d7b8f9c03389905c102)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:38.410787 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f104ef-92b4-4a5c-b379-4a30f4a4b2e6 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:38.648843 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-80f104ef-92b4-4a5c-b379-4a30f4a4b2e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6a8e1733f7e4d7b8f9c03389905c102'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:38.727864 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-80f104ef-92b4-4a5c-b379-4a30f4a4b2e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6a8e1733f7e4d7b8f9c03389905c102'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:38.728495 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-80f104ef-92b4-4a5c-b379-4a30f4a4b2e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6a8e1733f7e4d7b8f9c03389905c102'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:38.729111 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-80f104ef-92b4-4a5c-b379-4a30f4a4b2e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6a8e1733f7e4d7b8f9c03389905c102'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:38.739647 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-80f104ef-92b4-4a5c-b379-4a30f4a4b2e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6a8e1733f7e4d7b8f9c03389905c102'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:38.764163 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1328/2648] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:46:38 2022] DELETE /identity/v3/users/b6a8e1733f7e4d7b8f9c03389905c102 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:46:39.238776 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a1e433-4798-42c6-9c72-754a223890c0 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:39.246631 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-e0a1e433-4798-42c6-9c72-754a223890c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:46:39.253888 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a1e433-4798-42c6-9c72-754a223890c0 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:39.313941 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf320d11-786a-4872-a74d-1387dcd2d40c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:39.314573 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf320d11-786a-4872-a74d-1387dcd2d40c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:39.314852 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf320d11-786a-4872-a74d-1387dcd2d40c admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:39.315122 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf320d11-786a-4872-a74d-1387dcd2d40c admin admin] PATH_INFO: `/v3/projects/86d78bc517bc44f186c0eb1779a22dd6` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:39.316225 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf320d11-786a-4872-a74d-1387dcd2d40c admin admin] RBAC: Authorizing `identity:delete_project(project_id=86d78bc517bc44f186c0eb1779a22dd6)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:39.318912 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf320d11-786a-4872-a74d-1387dcd2d40c admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:39.448272 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-bf320d11-786a-4872-a74d-1387dcd2d40c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86d78bc517bc44f186c0eb1779a22dd6', 'request_id': 'req-bf320d11-786a-4872-a74d-1387dcd2d40c'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:39.455361 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-bf320d11-786a-4872-a74d-1387dcd2d40c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86d78bc517bc44f186c0eb1779a22dd6', 'request_id': 'req-bf320d11-786a-4872-a74d-1387dcd2d40c'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:39.522903 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-bf320d11-786a-4872-a74d-1387dcd2d40c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86d78bc517bc44f186c0eb1779a22dd6', 'request_id': 'req-bf320d11-786a-4872-a74d-1387dcd2d40c'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:39.537158 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-bf320d11-786a-4872-a74d-1387dcd2d40c admin admin] The token cache is being invalidate because project 86d78bc517bc44f186c0eb1779a22dd6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:46:39.537453 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-bf320d11-786a-4872-a74d-1387dcd2d40c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:39.538155 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-bf320d11-786a-4872-a74d-1387dcd2d40c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:39.539687 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1321/2649] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:46:39 2022] DELETE /identity/v3/projects/86d78bc517bc44f186c0eb1779a22dd6 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:46:39.546869 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f104ef-92b4-4a5c-b379-4a30f4a4b2e6 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:39.548457 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-80f104ef-92b4-4a5c-b379-4a30f4a4b2e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:46:39.561549 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f104ef-92b4-4a5c-b379-4a30f4a4b2e6 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:39.570326 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:46:39.620134 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387c72cc-3080-486d-8b85-d2be9edb10bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:39.620918 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387c72cc-3080-486d-8b85-d2be9edb10bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:39.621222 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387c72cc-3080-486d-8b85-d2be9edb10bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:39.621605 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387c72cc-3080-486d-8b85-d2be9edb10bb admin admin] PATH_INFO: `/v3/users/8273028285fe4cb4ae5f89a113765a7a` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:39.623166 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387c72cc-3080-486d-8b85-d2be9edb10bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=8273028285fe4cb4ae5f89a113765a7a)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:39.625838 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387c72cc-3080-486d-8b85-d2be9edb10bb admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:39.850082 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-387c72cc-3080-486d-8b85-d2be9edb10bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8273028285fe4cb4ae5f89a113765a7a'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:39.920147 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-387c72cc-3080-486d-8b85-d2be9edb10bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8273028285fe4cb4ae5f89a113765a7a'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:39.920776 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-387c72cc-3080-486d-8b85-d2be9edb10bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8273028285fe4cb4ae5f89a113765a7a'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:39.921387 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-387c72cc-3080-486d-8b85-d2be9edb10bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8273028285fe4cb4ae5f89a113765a7a'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:39.929322 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-387c72cc-3080-486d-8b85-d2be9edb10bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8273028285fe4cb4ae5f89a113765a7a'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:39.949831 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1329/2650] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:46:39 2022] DELETE /identity/v3/users/8273028285fe4cb4ae5f89a113765a7a => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:46:40.382058 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf320d11-786a-4872-a74d-1387dcd2d40c admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:40.384547 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-bf320d11-786a-4872-a74d-1387dcd2d40c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:46:40.398851 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf320d11-786a-4872-a74d-1387dcd2d40c admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:40.460069 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3320e4-1ef2-4691-9f6e-966b51065e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:40.460695 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3320e4-1ef2-4691-9f6e-966b51065e68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:40.460973 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3320e4-1ef2-4691-9f6e-966b51065e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:40.461245 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3320e4-1ef2-4691-9f6e-966b51065e68 admin admin] PATH_INFO: `/v3/projects/9114fb76157a47e78ae03e889a9db861` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:40.462389 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3320e4-1ef2-4691-9f6e-966b51065e68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9114fb76157a47e78ae03e889a9db861)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:40.464340 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3320e4-1ef2-4691-9f6e-966b51065e68 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:40.479371 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:46:40.591883 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-ae3320e4-1ef2-4691-9f6e-966b51065e68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9114fb76157a47e78ae03e889a9db861', 'request_id': 'req-ae3320e4-1ef2-4691-9f6e-966b51065e68'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:40.598011 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-ae3320e4-1ef2-4691-9f6e-966b51065e68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9114fb76157a47e78ae03e889a9db861', 'request_id': 'req-ae3320e4-1ef2-4691-9f6e-966b51065e68'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:40.693008 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-ae3320e4-1ef2-4691-9f6e-966b51065e68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9114fb76157a47e78ae03e889a9db861', 'request_id': 'req-ae3320e4-1ef2-4691-9f6e-966b51065e68'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:40.706358 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-ae3320e4-1ef2-4691-9f6e-966b51065e68 admin admin] The token cache is being invalidate because project 9114fb76157a47e78ae03e889a9db861 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:46:40.706902 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-ae3320e4-1ef2-4691-9f6e-966b51065e68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:40.707987 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-ae3320e4-1ef2-4691-9f6e-966b51065e68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:40.709760 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1322/2651] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:46:40 2022] DELETE /identity/v3/projects/9114fb76157a47e78ae03e889a9db861 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:46:40.714682 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8a5c0f-d830-4939-b901-e4207ef3d7b7 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:40.715147 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8a5c0f-d830-4939-b901-e4207ef3d7b7 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:40.715273 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8a5c0f-d830-4939-b901-e4207ef3d7b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:40.767683 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-ee8a5c0f-d830-4939-b901-e4207ef3d7b7 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:46:40.768926 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-ee8a5c0f-d830-4939-b901-e4207ef3d7b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:46:40.830364 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1330/2652] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:46:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:46:40.838932 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473e3f1a-bf84-4046-b53c-f4343c4a9b04 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:40.839218 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473e3f1a-bf84-4046-b53c-f4343c4a9b04 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:40.839491 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473e3f1a-bf84-4046-b53c-f4343c4a9b04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:40.863826 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-473e3f1a-bf84-4046-b53c-f4343c4a9b04 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:46:40.865049 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-473e3f1a-bf84-4046-b53c-f4343c4a9b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:46:40.873507 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1323/2653] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:46:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:46:40.877771 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8a5c0f-d830-4939-b901-e4207ef3d7b7 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:40.887289 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8a5c0f-d830-4939-b901-e4207ef3d7b7 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:40.892413 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7850b368-2f01-4c1e-80e8-9f396eeef3ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:40.893057 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7850b368-2f01-4c1e-80e8-9f396eeef3ea admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:40.893636 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7850b368-2f01-4c1e-80e8-9f396eeef3ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:40.894130 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7850b368-2f01-4c1e-80e8-9f396eeef3ea admin admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:40.894953 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7850b368-2f01-4c1e-80e8-9f396eeef3ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:46:40.895261 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7850b368-2f01-4c1e-80e8-9f396eeef3ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:40.897473 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7850b368-2f01-4c1e-80e8-9f396eeef3ea admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:40.901298 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:46:40.911159 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1331/2654] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 15 07:46:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:46:40.917328 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473e3f1a-bf84-4046-b53c-f4343c4a9b04 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:40.926940 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473e3f1a-bf84-4046-b53c-f4343c4a9b04 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:40.932038 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0aab81-eb8f-42a9-94d8-062cddb963df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:40.932702 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0aab81-eb8f-42a9-94d8-062cddb963df admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:40.932992 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0aab81-eb8f-42a9-94d8-062cddb963df admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:40.933274 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0aab81-eb8f-42a9-94d8-062cddb963df admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:40.933958 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0aab81-eb8f-42a9-94d8-062cddb963df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:40.936625 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0aab81-eb8f-42a9-94d8-062cddb963df admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:41.008879 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1324/2655] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:46:40 2022] POST /identity/v3/projects => generated 371 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 07:46:41.013929 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7850b368-2f01-4c1e-80e8-9f396eeef3ea admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:41.024170 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7850b368-2f01-4c1e-80e8-9f396eeef3ea admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:41.095465 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fd03f6-be76-42f5-9716-c2ad59be5c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:41.096080 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fd03f6-be76-42f5-9716-c2ad59be5c39 admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:41.096555 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fd03f6-be76-42f5-9716-c2ad59be5c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:41.096829 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fd03f6-be76-42f5-9716-c2ad59be5c39 admin admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:41.097467 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fd03f6-be76-42f5-9716-c2ad59be5c39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:41.099427 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fd03f6-be76-42f5-9716-c2ad59be5c39 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:41.179025 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1332/2656] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:46:41 2022] POST /identity/v3/users => generated 286 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:46:41.183233 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0aab81-eb8f-42a9-94d8-062cddb963df admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:41.194210 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0aab81-eb8f-42a9-94d8-062cddb963df admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:41.199721 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb4866c-15ce-4d3f-9955-61d8f282506c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:41.201549 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb4866c-15ce-4d3f-9955-61d8f282506c admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:41.201549 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb4866c-15ce-4d3f-9955-61d8f282506c admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:41.201549 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb4866c-15ce-4d3f-9955-61d8f282506c admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:41.202382 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb4866c-15ce-4d3f-9955-61d8f282506c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:41.204008 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb4866c-15ce-4d3f-9955-61d8f282506c admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:41.215369 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1325/2657] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:46:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:46:41.219804 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fd03f6-be76-42f5-9716-c2ad59be5c39 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:41.230060 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fd03f6-be76-42f5-9716-c2ad59be5c39 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:41.234761 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef76d84-27d7-4708-9a10-b67183486168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:41.235350 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef76d84-27d7-4708-9a10-b67183486168 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:41.235625 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef76d84-27d7-4708-9a10-b67183486168 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:41.235894 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef76d84-27d7-4708-9a10-b67183486168 admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:41.236523 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef76d84-27d7-4708-9a10-b67183486168 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:41.238122 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef76d84-27d7-4708-9a10-b67183486168 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:41.248113 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1333/2658] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:46:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:46:41.252314 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb4866c-15ce-4d3f-9955-61d8f282506c admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:41.264384 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb4866c-15ce-4d3f-9955-61d8f282506c admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:41.269125 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd0b448-7396-423e-bcb1-b363104f703e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:41.270810 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd0b448-7396-423e-bcb1-b363104f703e admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:41.270810 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd0b448-7396-423e-bcb1-b363104f703e admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:41.270810 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd0b448-7396-423e-bcb1-b363104f703e admin admin] PATH_INFO: `/v3/projects/988cf6a6fa2b452e8c4c017f282d144c/users/4d66f4fd90e74287ae9891e0364ab2cd/roles/053e12af84e943e699460a9d76c2072b` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:41.286284 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd0b448-7396-423e-bcb1-b363104f703e admin admin] RBAC: Authorizing `identity:create_grant(project_id=988cf6a6fa2b452e8c4c017f282d144c, user_id=4d66f4fd90e74287ae9891e0364ab2cd, role_id=053e12af84e943e699460a9d76c2072b)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:41.288899 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd0b448-7396-423e-bcb1-b363104f703e admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:41.360331 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1326/2659] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:46:41 2022] PUT /identity/v3/projects/988cf6a6fa2b452e8c4c017f282d144c/users/4d66f4fd90e74287ae9891e0364ab2cd/roles/053e12af84e943e699460a9d76c2072b => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 07:46:41.365386 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd74f24e-2b68-4082-9082-64bc06fe98e2 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:41.365705 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd74f24e-2b68-4082-9082-64bc06fe98e2 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:41.365988 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd74f24e-2b68-4082-9082-64bc06fe98e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:41.385188 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-fd74f24e-2b68-4082-9082-64bc06fe98e2 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:46:41.386451 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-fd74f24e-2b68-4082-9082-64bc06fe98e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:46:41.447106 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1334/2660] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:46:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:46:41.457242 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd0b448-7396-423e-bcb1-b363104f703e admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:41.458966 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-dbd0b448-7396-423e-bcb1-b363104f703e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:46:41.519397 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd0b448-7396-423e-bcb1-b363104f703e admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:41.523336 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c1f047-b7b2-4729-976e-47b42cffd23f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:41.524011 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c1f047-b7b2-4729-976e-47b42cffd23f service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:41.524288 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c1f047-b7b2-4729-976e-47b42cffd23f service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:41.524556 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c1f047-b7b2-4729-976e-47b42cffd23f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:41.535647 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c1f047-b7b2-4729-976e-47b42cffd23f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:41.537078 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c1f047-b7b2-4729-976e-47b42cffd23f service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:41.547679 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1327/2661] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:46:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:46:42.852708 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927479a1-1711-45d3-bb4f-6f9dd80b0c6e None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:42.852708 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927479a1-1711-45d3-bb4f-6f9dd80b0c6e None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:42.853290 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927479a1-1711-45d3-bb4f-6f9dd80b0c6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:42.884173 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-927479a1-1711-45d3-bb4f-6f9dd80b0c6e None None] MFA Rules not processed for user `4d66f4fd90e74287ae9891e0364ab2cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:46:42.885407 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-927479a1-1711-45d3-bb4f-6f9dd80b0c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:46:42.963369 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1335/2662] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:46:42 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:46:42.975563 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c1f047-b7b2-4729-976e-47b42cffd23f service neutron] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:42.977295 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-f9c1f047-b7b2-4729-976e-47b42cffd23f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:46:43.031395 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:46:43.047008 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c1f047-b7b2-4729-976e-47b42cffd23f service neutron] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:43.050078 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fe78d9-a718-4a00-a3fd-f71c8df527e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecf6642b7b84d3c9756bb41ab3349d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:43.050704 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fe78d9-a718-4a00-a3fd-f71c8df527e4 service cinder] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:43.050981 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fe78d9-a718-4a00-a3fd-f71c8df527e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:43.051263 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fe78d9-a718-4a00-a3fd-f71c8df527e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:43.061972 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fe78d9-a718-4a00-a3fd-f71c8df527e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:43.063419 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fe78d9-a718-4a00-a3fd-f71c8df527e4 service cinder] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:43.081666 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1328/2663] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:46:42 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:46:44.684816 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704eb333-c6e4-42c8-8e89-ece23d0bcde3 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:44.685349 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704eb333-c6e4-42c8-8e89-ece23d0bcde3 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:44.685475 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704eb333-c6e4-42c8-8e89-ece23d0bcde3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:44.693485 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:46:44.704745 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-704eb333-c6e4-42c8-8e89-ece23d0bcde3 None None] MFA Rules not processed for user `4d66f4fd90e74287ae9891e0364ab2cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:46:44.705982 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-704eb333-c6e4-42c8-8e89-ece23d0bcde3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:46:44.712822 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1336/2664] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:46:44 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:46:46.509222 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fe78d9-a718-4a00-a3fd-f71c8df527e4 service cinder] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:46.512576 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-f6fe78d9-a718-4a00-a3fd-f71c8df527e4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:46:46.602820 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fe78d9-a718-4a00-a3fd-f71c8df527e4 service cinder] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:46.607856 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f7a2eb-f2f3-43ee-afa6-bc9fbbef5d8c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd87846b9a134248b28e0521b0a880c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:46.608456 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f7a2eb-f2f3-43ee-afa6-bc9fbbef5d8c service placement] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:46.608736 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f7a2eb-f2f3-43ee-afa6-bc9fbbef5d8c service placement] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:46.609008 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f7a2eb-f2f3-43ee-afa6-bc9fbbef5d8c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:46.610808 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-27f7a2eb-f2f3-43ee-afa6-bc9fbbef5d8c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:46:46.622100 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f7a2eb-f2f3-43ee-afa6-bc9fbbef5d8c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:46.623650 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f7a2eb-f2f3-43ee-afa6-bc9fbbef5d8c service placement] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:46.633345 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1329/2665] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:46:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:46:53.424749 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704eb333-c6e4-42c8-8e89-ece23d0bcde3 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:53.426199 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-704eb333-c6e4-42c8-8e89-ece23d0bcde3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:46:53.439733 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704eb333-c6e4-42c8-8e89-ece23d0bcde3 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:53.444203 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ae0dd5-52f6-4553-a399-b5660575887e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:53.444836 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ae0dd5-52f6-4553-a399-b5660575887e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:53.445113 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ae0dd5-52f6-4553-a399-b5660575887e admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:53.445386 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ae0dd5-52f6-4553-a399-b5660575887e admin admin] PATH_INFO: `/v3/users/52f0cd498d8c49c58d7289c55188f5f4` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:53.446728 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ae0dd5-52f6-4553-a399-b5660575887e admin admin] RBAC: Authorizing `identity:delete_user(user_id=52f0cd498d8c49c58d7289c55188f5f4)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:53.448650 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ae0dd5-52f6-4553-a399-b5660575887e admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:53.636405 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-b4ae0dd5-52f6-4553-a399-b5660575887e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52f0cd498d8c49c58d7289c55188f5f4'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:53.685134 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-b4ae0dd5-52f6-4553-a399-b5660575887e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52f0cd498d8c49c58d7289c55188f5f4'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:53.685775 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-b4ae0dd5-52f6-4553-a399-b5660575887e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52f0cd498d8c49c58d7289c55188f5f4'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:53.686395 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-b4ae0dd5-52f6-4553-a399-b5660575887e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52f0cd498d8c49c58d7289c55188f5f4'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:53.694268 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-b4ae0dd5-52f6-4553-a399-b5660575887e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52f0cd498d8c49c58d7289c55188f5f4'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:53.712521 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1337/2666] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:46:53 2022] DELETE /identity/v3/users/52f0cd498d8c49c58d7289c55188f5f4 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:46:53.717440 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56749ba5-c73c-47cb-9272-6ac7f799121d None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:53.717972 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56749ba5-c73c-47cb-9272-6ac7f799121d None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:53.718084 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56749ba5-c73c-47cb-9272-6ac7f799121d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:53.737936 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-56749ba5-c73c-47cb-9272-6ac7f799121d None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:46:53.739166 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-56749ba5-c73c-47cb-9272-6ac7f799121d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:46:53.798151 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1330/2667] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:46:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 07:46:53.807863 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ae0dd5-52f6-4553-a399-b5660575887e admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:53.810321 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-b4ae0dd5-52f6-4553-a399-b5660575887e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:46:53.869628 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ae0dd5-52f6-4553-a399-b5660575887e admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:53.873929 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7b812e-de48-4c43-a6cc-347fe2603ba0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:53.880359 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7b812e-de48-4c43-a6cc-347fe2603ba0 service neutron] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:53.880359 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7b812e-de48-4c43-a6cc-347fe2603ba0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:53.880359 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7b812e-de48-4c43-a6cc-347fe2603ba0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:53.885058 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7b812e-de48-4c43-a6cc-347fe2603ba0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:53.886573 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7b812e-de48-4c43-a6cc-347fe2603ba0 service neutron] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:53.896887 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1338/2668] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:46:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:46:54.293123 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56749ba5-c73c-47cb-9272-6ac7f799121d None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:54.294604 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-56749ba5-c73c-47cb-9272-6ac7f799121d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:46:54.308243 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56749ba5-c73c-47cb-9272-6ac7f799121d None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:54.313605 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fe6301-0277-4251-8115-d8b7f294d724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:54.314404 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fe6301-0277-4251-8115-d8b7f294d724 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:54.314676 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fe6301-0277-4251-8115-d8b7f294d724 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:54.314948 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fe6301-0277-4251-8115-d8b7f294d724 admin admin] PATH_INFO: `/v3/projects/e681e2f82201457eb7fb7e753807f155` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:54.316084 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fe6301-0277-4251-8115-d8b7f294d724 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e681e2f82201457eb7fb7e753807f155)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:54.318044 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fe6301-0277-4251-8115-d8b7f294d724 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:54.405290 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:46:54.406819 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-a9fe6301-0277-4251-8115-d8b7f294d724 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e681e2f82201457eb7fb7e753807f155', 'request_id': 'req-a9fe6301-0277-4251-8115-d8b7f294d724'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:54.413974 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-a9fe6301-0277-4251-8115-d8b7f294d724 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e681e2f82201457eb7fb7e753807f155', 'request_id': 'req-a9fe6301-0277-4251-8115-d8b7f294d724'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:54.462314 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-a9fe6301-0277-4251-8115-d8b7f294d724 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e681e2f82201457eb7fb7e753807f155', 'request_id': 'req-a9fe6301-0277-4251-8115-d8b7f294d724'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:54.473357 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-a9fe6301-0277-4251-8115-d8b7f294d724 admin admin] The token cache is being invalidate because project e681e2f82201457eb7fb7e753807f155 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:46:54.474218 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-a9fe6301-0277-4251-8115-d8b7f294d724 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:54.475216 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-a9fe6301-0277-4251-8115-d8b7f294d724 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:46:54.476708 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1331/2669] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:46:54 2022] DELETE /identity/v3/projects/e681e2f82201457eb7fb7e753807f155 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:46:54.482293 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4ed217-f929-42be-9b6e-7f8bda15a395 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:54.482761 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4ed217-f929-42be-9b6e-7f8bda15a395 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:54.482886 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4ed217-f929-42be-9b6e-7f8bda15a395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:54.503158 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-cf4ed217-f929-42be-9b6e-7f8bda15a395 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:46:54.504352 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-cf4ed217-f929-42be-9b6e-7f8bda15a395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:46:54.565907 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1339/2670] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:46:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:46:54.575296 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b30b482-76f1-412c-9b2a-91f4c69f0f51 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:54.575616 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b30b482-76f1-412c-9b2a-91f4c69f0f51 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:54.575900 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b30b482-76f1-412c-9b2a-91f4c69f0f51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:54.596975 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-1b30b482-76f1-412c-9b2a-91f4c69f0f51 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:46:54.598256 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-1b30b482-76f1-412c-9b2a-91f4c69f0f51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:46:54.607046 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1332/2671] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:46:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:46:54.611363 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4ed217-f929-42be-9b6e-7f8bda15a395 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:54.622306 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4ed217-f929-42be-9b6e-7f8bda15a395 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:54.627212 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270a91a2-41ff-475f-8008-5ea54db8dcfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:54.627824 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270a91a2-41ff-475f-8008-5ea54db8dcfd admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:54.628106 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270a91a2-41ff-475f-8008-5ea54db8dcfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:54.628376 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270a91a2-41ff-475f-8008-5ea54db8dcfd admin admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:54.629049 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270a91a2-41ff-475f-8008-5ea54db8dcfd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:46:54.629356 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270a91a2-41ff-475f-8008-5ea54db8dcfd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:54.630978 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270a91a2-41ff-475f-8008-5ea54db8dcfd admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:54.640237 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:46:54.643740 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1340/2672] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 15 07:46:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:46:54.648129 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b30b482-76f1-412c-9b2a-91f4c69f0f51 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:54.657688 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b30b482-76f1-412c-9b2a-91f4c69f0f51 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:54.662536 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb75574f-fec0-41fa-b89b-a1b5f97d6575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:54.663139 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb75574f-fec0-41fa-b89b-a1b5f97d6575 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:54.663415 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb75574f-fec0-41fa-b89b-a1b5f97d6575 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:54.663684 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb75574f-fec0-41fa-b89b-a1b5f97d6575 admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:54.664321 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb75574f-fec0-41fa-b89b-a1b5f97d6575 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:54.666219 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb75574f-fec0-41fa-b89b-a1b5f97d6575 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:54.767179 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1333/2673] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:46:54 2022] POST /identity/v3/projects => generated 391 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:46:54.771386 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270a91a2-41ff-475f-8008-5ea54db8dcfd admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:54.781694 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270a91a2-41ff-475f-8008-5ea54db8dcfd admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:54.839063 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c37673-177e-4b73-83ec-68002b91ee81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:54.839677 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c37673-177e-4b73-83ec-68002b91ee81 admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:54.840022 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c37673-177e-4b73-83ec-68002b91ee81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:54.840281 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c37673-177e-4b73-83ec-68002b91ee81 admin admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:54.841003 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c37673-177e-4b73-83ec-68002b91ee81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:54.842953 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c37673-177e-4b73-83ec-68002b91ee81 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:54.902185 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1341/2674] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:46:54 2022] POST /identity/v3/users => generated 296 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:46:54.906959 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb75574f-fec0-41fa-b89b-a1b5f97d6575 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:54.917126 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb75574f-fec0-41fa-b89b-a1b5f97d6575 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:54.922322 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f565bc-a976-44f4-a747-335c1058ad2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:54.922971 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f565bc-a976-44f4-a747-335c1058ad2e admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:54.923195 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f565bc-a976-44f4-a747-335c1058ad2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:54.923470 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f565bc-a976-44f4-a747-335c1058ad2e admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:54.924106 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f565bc-a976-44f4-a747-335c1058ad2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:54.925724 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f565bc-a976-44f4-a747-335c1058ad2e admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:54.936141 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1334/2675] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:46:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:46:54.940029 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c37673-177e-4b73-83ec-68002b91ee81 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:54.950531 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c37673-177e-4b73-83ec-68002b91ee81 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:54.955240 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8656f4-4239-449a-9c40-62627dfe61ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:54.955871 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8656f4-4239-449a-9c40-62627dfe61ac admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:54.956151 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8656f4-4239-449a-9c40-62627dfe61ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:54.956423 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8656f4-4239-449a-9c40-62627dfe61ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:54.957055 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8656f4-4239-449a-9c40-62627dfe61ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:54.958687 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8656f4-4239-449a-9c40-62627dfe61ac admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:54.970119 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1342/2676] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:46:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:46:54.974243 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f565bc-a976-44f4-a747-335c1058ad2e admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:54.984653 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f565bc-a976-44f4-a747-335c1058ad2e admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:54.989293 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8574654a-900b-43e9-9ee9-f7912276f022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:54.989967 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8574654a-900b-43e9-9ee9-f7912276f022 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:54.990247 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8574654a-900b-43e9-9ee9-f7912276f022 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:54.990520 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8574654a-900b-43e9-9ee9-f7912276f022 admin admin] PATH_INFO: `/v3/projects/1a44ea4ae3d04e75982dac9479986bf7/users/0b9377cc49db44bc9a62c1d22e5d972e/roles/053e12af84e943e699460a9d76c2072b` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:55.004825 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8574654a-900b-43e9-9ee9-f7912276f022 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a44ea4ae3d04e75982dac9479986bf7, user_id=0b9377cc49db44bc9a62c1d22e5d972e, role_id=053e12af84e943e699460a9d76c2072b)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:55.007463 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8574654a-900b-43e9-9ee9-f7912276f022 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:55.057377 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8656f4-4239-449a-9c40-62627dfe61ac admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:55.059375 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-4b8656f4-4239-449a-9c40-62627dfe61ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:46:55.072559 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1335/2677] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:46:54 2022] PUT /identity/v3/projects/1a44ea4ae3d04e75982dac9479986bf7/users/0b9377cc49db44bc9a62c1d22e5d972e/roles/053e12af84e943e699460a9d76c2072b => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 07:46:55.082979 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daff2b3c-0225-4497-849c-8f76b02b2ee8 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:55.083262 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daff2b3c-0225-4497-849c-8f76b02b2ee8 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:55.083541 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daff2b3c-0225-4497-849c-8f76b02b2ee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:55.116545 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-daff2b3c-0225-4497-849c-8f76b02b2ee8 None None] MFA Rules not processed for user `0b9377cc49db44bc9a62c1d22e5d972e`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:46:55.117959 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-daff2b3c-0225-4497-849c-8f76b02b2ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:46:55.131299 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8656f4-4239-449a-9c40-62627dfe61ac admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:55.195999 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1336/2678] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:46:55 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 07:46:55.207022 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daff2b3c-0225-4497-849c-8f76b02b2ee8 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:46:55.208459 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-daff2b3c-0225-4497-849c-8f76b02b2ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:46:55.269732 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daff2b3c-0225-4497-849c-8f76b02b2ee8 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:46:55.273523 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b88aad8-259c-4f54-93d4-3f1e60a24123 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecf6642b7b84d3c9756bb41ab3349d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:55.274348 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b88aad8-259c-4f54-93d4-3f1e60a24123 service cinder] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:55.274634 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b88aad8-259c-4f54-93d4-3f1e60a24123 service cinder] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:55.274944 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b88aad8-259c-4f54-93d4-3f1e60a24123 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:55.284111 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b88aad8-259c-4f54-93d4-3f1e60a24123 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:55.285537 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b88aad8-259c-4f54-93d4-3f1e60a24123 service cinder] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:55.294991 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1337/2679] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:46:55 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 07:46:55.306243 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749bf188-122a-4244-8677-ad07f5d5d2f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:46:55.306947 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749bf188-122a-4244-8677-ad07f5d5d2f9 service neutron] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:46:55.307227 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749bf188-122a-4244-8677-ad07f5d5d2f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:46:55.308138 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749bf188-122a-4244-8677-ad07f5d5d2f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:46:55.310201 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-749bf188-122a-4244-8677-ad07f5d5d2f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:46:55.358805 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:46:55.369770 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749bf188-122a-4244-8677-ad07f5d5d2f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:46:55.371827 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749bf188-122a-4244-8677-ad07f5d5d2f9 service neutron] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:46:55.381293 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1343/2680] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:46:55 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 07:47:05.290459 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:47:05.428715 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=5 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:47:06.712718 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b88aad8-259c-4f54-93d4-3f1e60a24123 service cinder] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:47:06.715214 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-3b88aad8-259c-4f54-93d4-3f1e60a24123 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:47:06.749293 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b88aad8-259c-4f54-93d4-3f1e60a24123 service cinder] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:47:06.752875 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79e8393-a39a-4095-be0d-af3bba55029e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecf6642b7b84d3c9756bb41ab3349d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:47:06.753478 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79e8393-a39a-4095-be0d-af3bba55029e service cinder] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:47:06.753789 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79e8393-a39a-4095-be0d-af3bba55029e service cinder] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:47:06.754066 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79e8393-a39a-4095-be0d-af3bba55029e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:47:06.755761 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-a79e8393-a39a-4095-be0d-af3bba55029e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:47:06.816395 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79e8393-a39a-4095-be0d-af3bba55029e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:47:06.817846 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79e8393-a39a-4095-be0d-af3bba55029e service cinder] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:47:06.827074 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1338/2681] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:47:06 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:47:13.628645 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749bf188-122a-4244-8677-ad07f5d5d2f9 service neutron] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:47:13.646942 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749bf188-122a-4244-8677-ad07f5d5d2f9 service neutron] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:47:13.651095 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4ac5d8-4d03-4a06-8daf-59c13d57fd4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:47:13.651701 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4ac5d8-4d03-4a06-8daf-59c13d57fd4f service neutron] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:47:13.651977 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4ac5d8-4d03-4a06-8daf-59c13d57fd4f service neutron] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:47:13.652247 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4ac5d8-4d03-4a06-8daf-59c13d57fd4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:47:13.654036 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-fb4ac5d8-4d03-4a06-8daf-59c13d57fd4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:47:13.717428 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4ac5d8-4d03-4a06-8daf-59c13d57fd4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:47:13.718943 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4ac5d8-4d03-4a06-8daf-59c13d57fd4f service neutron] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:47:13.729263 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1344/2682] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:47:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:47:14.364298 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dad2026-9295-4cf6-aeb9-13803db95db5 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:47:14.365130 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dad2026-9295-4cf6-aeb9-13803db95db5 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:47:14.365130 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dad2026-9295-4cf6-aeb9-13803db95db5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:47:14.384446 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-7dad2026-9295-4cf6-aeb9-13803db95db5 None None] MFA Rules not processed for user `4d66f4fd90e74287ae9891e0364ab2cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:47:14.386093 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-7dad2026-9295-4cf6-aeb9-13803db95db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:47:14.455645 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1339/2683] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:47:14 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:47:16.460598 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1a0a93-3470-4cb7-96c5-982bb5616561 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:47:16.461144 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1a0a93-3470-4cb7-96c5-982bb5616561 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:47:16.461144 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1a0a93-3470-4cb7-96c5-982bb5616561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:47:16.480986 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-5a1a0a93-3470-4cb7-96c5-982bb5616561 None None] MFA Rules not processed for user `ebd85e42855448268c89a200f4392316`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:47:16.482217 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-5a1a0a93-3470-4cb7-96c5-982bb5616561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:47:16.489143 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1345/2684] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:47:16 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:47:16.805191 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dad2026-9295-4cf6-aeb9-13803db95db5 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:47:16.818732 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dad2026-9295-4cf6-aeb9-13803db95db5 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:47:16.824659 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b479c3c0-9ca5-42c7-9f8b-7c4c2a2ac3b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:47:16.825718 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b479c3c0-9ca5-42c7-9f8b-7c4c2a2ac3b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:47:16.834889 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b479c3c0-9ca5-42c7-9f8b-7c4c2a2ac3b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:47:16.838184 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b479c3c0-9ca5-42c7-9f8b-7c4c2a2ac3b8 admin admin] PATH_INFO: `/v3/users/0b9377cc49db44bc9a62c1d22e5d972e` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:47:16.840784 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b479c3c0-9ca5-42c7-9f8b-7c4c2a2ac3b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b9377cc49db44bc9a62c1d22e5d972e)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:47:16.842997 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b479c3c0-9ca5-42c7-9f8b-7c4c2a2ac3b8 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:47:17.131391 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-b479c3c0-9ca5-42c7-9f8b-7c4c2a2ac3b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b9377cc49db44bc9a62c1d22e5d972e'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:47:17.133495 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-b479c3c0-9ca5-42c7-9f8b-7c4c2a2ac3b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b9377cc49db44bc9a62c1d22e5d972e'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:47:17.206441 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-b479c3c0-9ca5-42c7-9f8b-7c4c2a2ac3b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b9377cc49db44bc9a62c1d22e5d972e'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:47:17.207185 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-b479c3c0-9ca5-42c7-9f8b-7c4c2a2ac3b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b9377cc49db44bc9a62c1d22e5d972e'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:47:17.222185 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-b479c3c0-9ca5-42c7-9f8b-7c4c2a2ac3b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b9377cc49db44bc9a62c1d22e5d972e'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:47:17.245110 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:47:17.253716 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1340/2685] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:47:16 2022] DELETE /identity/v3/users/0b9377cc49db44bc9a62c1d22e5d972e => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:47:17.258085 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cee491-7a7b-426f-9892-fe93d3c9b8a5 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:47:17.261953 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cee491-7a7b-426f-9892-fe93d3c9b8a5 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:47:17.261953 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cee491-7a7b-426f-9892-fe93d3c9b8a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:47:17.295127 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-60cee491-7a7b-426f-9892-fe93d3c9b8a5 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:47:17.296820 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-60cee491-7a7b-426f-9892-fe93d3c9b8a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:47:17.406843 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1346/2686] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:47:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 07:47:17.426020 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b479c3c0-9ca5-42c7-9f8b-7c4c2a2ac3b8 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:47:17.432495 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-b479c3c0-9ca5-42c7-9f8b-7c4c2a2ac3b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:47:17.599917 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b479c3c0-9ca5-42c7-9f8b-7c4c2a2ac3b8 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:47:17.606454 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb375f8-1ee9-42d9-8a20-7349c810fda3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:47:17.606454 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb375f8-1ee9-42d9-8a20-7349c810fda3 service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:47:17.606454 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb375f8-1ee9-42d9-8a20-7349c810fda3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:47:17.606454 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb375f8-1ee9-42d9-8a20-7349c810fda3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:47:17.616467 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb375f8-1ee9-42d9-8a20-7349c810fda3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:47:17.617846 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb375f8-1ee9-42d9-8a20-7349c810fda3 service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:47:17.632258 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1341/2687] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:47:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:47:18.568057 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cee491-7a7b-426f-9892-fe93d3c9b8a5 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:47:18.569676 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-60cee491-7a7b-426f-9892-fe93d3c9b8a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:47:18.586792 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cee491-7a7b-426f-9892-fe93d3c9b8a5 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:47:18.593004 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc777c0-2ccf-497f-9a22-02982f1a888b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:47:18.594550 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc777c0-2ccf-497f-9a22-02982f1a888b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:47:18.595104 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc777c0-2ccf-497f-9a22-02982f1a888b admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:47:18.595555 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc777c0-2ccf-497f-9a22-02982f1a888b admin admin] PATH_INFO: `/v3/projects/1a44ea4ae3d04e75982dac9479986bf7` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:47:18.597451 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc777c0-2ccf-497f-9a22-02982f1a888b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a44ea4ae3d04e75982dac9479986bf7)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:47:18.601050 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc777c0-2ccf-497f-9a22-02982f1a888b admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:47:18.754147 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-8fc777c0-2ccf-497f-9a22-02982f1a888b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a44ea4ae3d04e75982dac9479986bf7', 'request_id': 'req-8fc777c0-2ccf-497f-9a22-02982f1a888b'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:47:18.763811 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-8fc777c0-2ccf-497f-9a22-02982f1a888b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a44ea4ae3d04e75982dac9479986bf7', 'request_id': 'req-8fc777c0-2ccf-497f-9a22-02982f1a888b'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:47:18.832317 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-8fc777c0-2ccf-497f-9a22-02982f1a888b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a44ea4ae3d04e75982dac9479986bf7', 'request_id': 'req-8fc777c0-2ccf-497f-9a22-02982f1a888b'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:47:18.852603 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-8fc777c0-2ccf-497f-9a22-02982f1a888b admin admin] The token cache is being invalidate because project 1a44ea4ae3d04e75982dac9479986bf7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:47:18.854710 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-8fc777c0-2ccf-497f-9a22-02982f1a888b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:47:18.855448 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-8fc777c0-2ccf-497f-9a22-02982f1a888b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:47:18.858406 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1347/2688] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:47:18 2022] DELETE /identity/v3/projects/1a44ea4ae3d04e75982dac9479986bf7 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:47:18.863224 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461a8d72-6162-4bc3-90cd-ea88c2bc7a2f None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:47:18.864227 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461a8d72-6162-4bc3-90cd-ea88c2bc7a2f None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:47:18.864669 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461a8d72-6162-4bc3-90cd-ea88c2bc7a2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:47:18.891651 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-461a8d72-6162-4bc3-90cd-ea88c2bc7a2f None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:47:18.893283 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-461a8d72-6162-4bc3-90cd-ea88c2bc7a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:47:18.979452 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1342/2689] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:47:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:47:18.981305 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a412b2-da0c-4dd5-b2e2-cccc802d65e7 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:47:18.981603 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a412b2-da0c-4dd5-b2e2-cccc802d65e7 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:47:18.981953 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a412b2-da0c-4dd5-b2e2-cccc802d65e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:47:18.991024 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:47:19.005820 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-a3a412b2-da0c-4dd5-b2e2-cccc802d65e7 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:47:19.007147 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-a3a412b2-da0c-4dd5-b2e2-cccc802d65e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:47:19.020527 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1348/2690] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:47:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:47:19.022011 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461a8d72-6162-4bc3-90cd-ea88c2bc7a2f None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:47:19.051194 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461a8d72-6162-4bc3-90cd-ea88c2bc7a2f None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:47:19.056701 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2a8911-4990-41f5-8951-1c94e785eeca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:47:19.057404 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2a8911-4990-41f5-8951-1c94e785eeca admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:47:19.057772 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2a8911-4990-41f5-8951-1c94e785eeca admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:47:19.058045 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2a8911-4990-41f5-8951-1c94e785eeca admin admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:47:19.058704 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2a8911-4990-41f5-8951-1c94e785eeca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:47:19.059011 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2a8911-4990-41f5-8951-1c94e785eeca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:47:19.060571 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2a8911-4990-41f5-8951-1c94e785eeca admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:47:19.077499 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1343/2691] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 15 07:47:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:47:19.078826 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a412b2-da0c-4dd5-b2e2-cccc802d65e7 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:47:19.088269 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a412b2-da0c-4dd5-b2e2-cccc802d65e7 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:47:19.093156 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec15bd1c-89f9-4d7f-afdb-1abcc71fb00f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:47:19.093797 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec15bd1c-89f9-4d7f-afdb-1abcc71fb00f admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:47:19.094083 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec15bd1c-89f9-4d7f-afdb-1abcc71fb00f admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:47:19.094355 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec15bd1c-89f9-4d7f-afdb-1abcc71fb00f admin admin] PATH_INFO: `/v3/projects` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:47:19.095165 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec15bd1c-89f9-4d7f-afdb-1abcc71fb00f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:47:19.097339 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec15bd1c-89f9-4d7f-afdb-1abcc71fb00f admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:47:19.225411 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1349/2692] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:47:19 2022] POST /identity/v3/projects => generated 385 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:47:19.231803 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2a8911-4990-41f5-8951-1c94e785eeca admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:47:19.241050 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2a8911-4990-41f5-8951-1c94e785eeca admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:47:19.318470 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f962d1-c3e2-4269-b6e6-0c4455c4de91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:47:19.324973 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f962d1-c3e2-4269-b6e6-0c4455c4de91 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:47:19.324973 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f962d1-c3e2-4269-b6e6-0c4455c4de91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:47:19.324973 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f962d1-c3e2-4269-b6e6-0c4455c4de91 admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:47:19.324973 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f962d1-c3e2-4269-b6e6-0c4455c4de91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:47:19.324973 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f962d1-c3e2-4269-b6e6-0c4455c4de91 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:47:19.416776 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1344/2693] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:47:19 2022] POST /identity/v3/users => generated 293 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:47:19.420541 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec15bd1c-89f9-4d7f-afdb-1abcc71fb00f admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:47:19.437617 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec15bd1c-89f9-4d7f-afdb-1abcc71fb00f admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:47:19.443666 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bf31e1-3578-4c0b-aada-e9e6491bfae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:47:19.447244 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bf31e1-3578-4c0b-aada-e9e6491bfae5 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:47:19.447244 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bf31e1-3578-4c0b-aada-e9e6491bfae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:47:19.447244 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bf31e1-3578-4c0b-aada-e9e6491bfae5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:47:19.447244 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bf31e1-3578-4c0b-aada-e9e6491bfae5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:47:19.449971 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bf31e1-3578-4c0b-aada-e9e6491bfae5 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:47:19.468904 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1350/2694] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:47:19 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:47:19.470229 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f962d1-c3e2-4269-b6e6-0c4455c4de91 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:47:19.479362 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:47:19.484293 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f962d1-c3e2-4269-b6e6-0c4455c4de91 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:47:19.498123 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de2e54f-4c2d-4d25-ab3f-a5f74fa49835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:47:19.504045 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de2e54f-4c2d-4d25-ab3f-a5f74fa49835 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:47:19.504045 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de2e54f-4c2d-4d25-ab3f-a5f74fa49835 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:47:19.504045 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de2e54f-4c2d-4d25-ab3f-a5f74fa49835 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:47:19.504045 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de2e54f-4c2d-4d25-ab3f-a5f74fa49835 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:47:19.504045 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de2e54f-4c2d-4d25-ab3f-a5f74fa49835 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:47:19.515799 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1345/2695] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:47:19 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:47:19.519868 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bf31e1-3578-4c0b-aada-e9e6491bfae5 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:47:19.533182 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bf31e1-3578-4c0b-aada-e9e6491bfae5 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:47:19.539414 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce8cdd5-0cd0-44a5-86ba-088626ea67fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:47:19.540432 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce8cdd5-0cd0-44a5-86ba-088626ea67fc admin admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:47:19.541303 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce8cdd5-0cd0-44a5-86ba-088626ea67fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:47:19.541787 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce8cdd5-0cd0-44a5-86ba-088626ea67fc admin admin] PATH_INFO: `/v3/projects/00091272c5a745f2bf80723a69fbfc61/users/f4c762318a844714b585f80fdfa9ef53/roles/053e12af84e943e699460a9d76c2072b` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:47:19.561327 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce8cdd5-0cd0-44a5-86ba-088626ea67fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=00091272c5a745f2bf80723a69fbfc61, user_id=f4c762318a844714b585f80fdfa9ef53, role_id=053e12af84e943e699460a9d76c2072b)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:47:19.564506 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce8cdd5-0cd0-44a5-86ba-088626ea67fc admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:47:19.692461 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1351/2696] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:47:19 2022] PUT /identity/v3/projects/00091272c5a745f2bf80723a69fbfc61/users/f4c762318a844714b585f80fdfa9ef53/roles/053e12af84e943e699460a9d76c2072b => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:47:19.700904 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede56afa-44cf-435f-9536-9b8816ccb5a4 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:47:19.700904 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede56afa-44cf-435f-9536-9b8816ccb5a4 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:47:19.700904 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede56afa-44cf-435f-9536-9b8816ccb5a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:47:19.715265 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-ede56afa-44cf-435f-9536-9b8816ccb5a4 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:47:19.716599 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-ede56afa-44cf-435f-9536-9b8816ccb5a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:47:19.792188 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1346/2697] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:47:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 07:47:19.802935 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce8cdd5-0cd0-44a5-86ba-088626ea67fc admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:47:19.805168 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-3ce8cdd5-0cd0-44a5-86ba-088626ea67fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:47:19.873527 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce8cdd5-0cd0-44a5-86ba-088626ea67fc admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:47:19.878560 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbf5338-1546-4b3e-91be-94f28def1158 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:47:19.879308 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbf5338-1546-4b3e-91be-94f28def1158 service neutron] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:47:19.880034 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbf5338-1546-4b3e-91be-94f28def1158 service neutron] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:47:19.880163 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbf5338-1546-4b3e-91be-94f28def1158 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:47:19.901976 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbf5338-1546-4b3e-91be-94f28def1158 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:47:19.901976 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbf5338-1546-4b3e-91be-94f28def1158 service neutron] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:47:19.923228 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1352/2698] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:47:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:47:24.040019 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede56afa-44cf-435f-9536-9b8816ccb5a4 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:47:24.043290 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-ede56afa-44cf-435f-9536-9b8816ccb5a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:47:24.059476 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede56afa-44cf-435f-9536-9b8816ccb5a4 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:47:24.073417 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254f178d-d119-4107-a415-398b3f3fc34e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:47:24.074271 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254f178d-d119-4107-a415-398b3f3fc34e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:47:24.077775 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254f178d-d119-4107-a415-398b3f3fc34e admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:47:24.078172 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254f178d-d119-4107-a415-398b3f3fc34e admin admin] PATH_INFO: `/v3/users/0a1410abac494fd18c859268b528ec0c` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:47:24.087363 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254f178d-d119-4107-a415-398b3f3fc34e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a1410abac494fd18c859268b528ec0c)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:47:24.089866 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254f178d-d119-4107-a415-398b3f3fc34e admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:47:24.365696 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-254f178d-d119-4107-a415-398b3f3fc34e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a1410abac494fd18c859268b528ec0c'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:47:24.366394 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-254f178d-d119-4107-a415-398b3f3fc34e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a1410abac494fd18c859268b528ec0c'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:47:24.475930 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-254f178d-d119-4107-a415-398b3f3fc34e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a1410abac494fd18c859268b528ec0c'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:47:24.476567 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-254f178d-d119-4107-a415-398b3f3fc34e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a1410abac494fd18c859268b528ec0c'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:47:24.485767 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-254f178d-d119-4107-a415-398b3f3fc34e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a1410abac494fd18c859268b528ec0c'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:47:24.518576 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1347/2699] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:47:24 2022] DELETE /identity/v3/users/0a1410abac494fd18c859268b528ec0c => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:47:24.848164 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbf5338-1546-4b3e-91be-94f28def1158 service neutron] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:47:24.862384 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbf5338-1546-4b3e-91be-94f28def1158 service neutron] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:47:24.937366 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fefb97-6645-41ec-97af-72e25c3a1dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:47:24.938237 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fefb97-6645-41ec-97af-72e25c3a1dcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:47:24.938641 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fefb97-6645-41ec-97af-72e25c3a1dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:47:24.939175 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fefb97-6645-41ec-97af-72e25c3a1dcd admin admin] PATH_INFO: `/v3/projects/7e20a6e5fded473c8905bffff7775ed3` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:47:24.941982 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fefb97-6645-41ec-97af-72e25c3a1dcd admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e20a6e5fded473c8905bffff7775ed3)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:47:24.944533 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fefb97-6645-41ec-97af-72e25c3a1dcd admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:47:25.078594 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:47:25.082616 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-b1fefb97-6645-41ec-97af-72e25c3a1dcd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e20a6e5fded473c8905bffff7775ed3', 'request_id': 'req-b1fefb97-6645-41ec-97af-72e25c3a1dcd'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:47:25.091250 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-b1fefb97-6645-41ec-97af-72e25c3a1dcd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e20a6e5fded473c8905bffff7775ed3', 'request_id': 'req-b1fefb97-6645-41ec-97af-72e25c3a1dcd'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:47:25.171329 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-b1fefb97-6645-41ec-97af-72e25c3a1dcd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e20a6e5fded473c8905bffff7775ed3', 'request_id': 'req-b1fefb97-6645-41ec-97af-72e25c3a1dcd'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:47:25.186216 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-b1fefb97-6645-41ec-97af-72e25c3a1dcd admin admin] The token cache is being invalidate because project 7e20a6e5fded473c8905bffff7775ed3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:47:25.186520 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-b1fefb97-6645-41ec-97af-72e25c3a1dcd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:47:25.187176 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-b1fefb97-6645-41ec-97af-72e25c3a1dcd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:47:25.189108 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1353/2700] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:47:24 2022] DELETE /identity/v3/projects/7e20a6e5fded473c8905bffff7775ed3 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:47:25.193135 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254f178d-d119-4107-a415-398b3f3fc34e admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:47:25.194629 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-254f178d-d119-4107-a415-398b3f3fc34e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:47:25.208551 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254f178d-d119-4107-a415-398b3f3fc34e admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:47:25.285237 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24721c3f-ad71-4fae-b8c6-061048f6abb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:47:25.285917 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24721c3f-ad71-4fae-b8c6-061048f6abb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:47:25.286202 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24721c3f-ad71-4fae-b8c6-061048f6abb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:47:25.286480 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24721c3f-ad71-4fae-b8c6-061048f6abb9 admin admin] PATH_INFO: `/v3/users/b07de6d68f1140c8a7e1c893bf2abdd8` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:47:25.292645 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24721c3f-ad71-4fae-b8c6-061048f6abb9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b07de6d68f1140c8a7e1c893bf2abdd8)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:47:25.292645 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24721c3f-ad71-4fae-b8c6-061048f6abb9 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:47:25.295589 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:47:25.500437 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-24721c3f-ad71-4fae-b8c6-061048f6abb9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b07de6d68f1140c8a7e1c893bf2abdd8'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:47:25.501188 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-24721c3f-ad71-4fae-b8c6-061048f6abb9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b07de6d68f1140c8a7e1c893bf2abdd8'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:47:25.572735 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-24721c3f-ad71-4fae-b8c6-061048f6abb9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b07de6d68f1140c8a7e1c893bf2abdd8'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:47:25.573426 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-24721c3f-ad71-4fae-b8c6-061048f6abb9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b07de6d68f1140c8a7e1c893bf2abdd8'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:47:25.581298 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-24721c3f-ad71-4fae-b8c6-061048f6abb9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b07de6d68f1140c8a7e1c893bf2abdd8'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:47:25.599813 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1348/2701] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:47:25 2022] DELETE /identity/v3/users/b07de6d68f1140c8a7e1c893bf2abdd8 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:47:25.884037 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fefb97-6645-41ec-97af-72e25c3a1dcd admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:47:25.886138 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-b1fefb97-6645-41ec-97af-72e25c3a1dcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:47:25.899558 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fefb97-6645-41ec-97af-72e25c3a1dcd admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:47:25.971669 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3a2c57-8545-48f6-8509-ea0ea220e7ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:47:25.974194 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3a2c57-8545-48f6-8509-ea0ea220e7ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:47:25.974194 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3a2c57-8545-48f6-8509-ea0ea220e7ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:47:25.974194 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3a2c57-8545-48f6-8509-ea0ea220e7ff admin admin] PATH_INFO: `/v3/projects/c1f22762fda74145b0fd8abb6d133392` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:47:25.974778 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3a2c57-8545-48f6-8509-ea0ea220e7ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1f22762fda74145b0fd8abb6d133392)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:47:25.977928 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3a2c57-8545-48f6-8509-ea0ea220e7ff admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:47:26.171149 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-8b3a2c57-8545-48f6-8509-ea0ea220e7ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1f22762fda74145b0fd8abb6d133392', 'request_id': 'req-8b3a2c57-8545-48f6-8509-ea0ea220e7ff'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:47:26.181176 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-8b3a2c57-8545-48f6-8509-ea0ea220e7ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1f22762fda74145b0fd8abb6d133392', 'request_id': 'req-8b3a2c57-8545-48f6-8509-ea0ea220e7ff'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:47:26.241516 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-8b3a2c57-8545-48f6-8509-ea0ea220e7ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1f22762fda74145b0fd8abb6d133392', 'request_id': 'req-8b3a2c57-8545-48f6-8509-ea0ea220e7ff'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:47:26.256551 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-8b3a2c57-8545-48f6-8509-ea0ea220e7ff admin admin] The token cache is being invalidate because project c1f22762fda74145b0fd8abb6d133392 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:47:26.256940 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-8b3a2c57-8545-48f6-8509-ea0ea220e7ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:47:26.257730 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-8b3a2c57-8545-48f6-8509-ea0ea220e7ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:47:26.260299 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1354/2702] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:47:25 2022] DELETE /identity/v3/projects/c1f22762fda74145b0fd8abb6d133392 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:47:28.252561 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559113d2-8044-42f4-a530-5eba2b612c93 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:47:28.253642 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559113d2-8044-42f4-a530-5eba2b612c93 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:47:28.254111 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559113d2-8044-42f4-a530-5eba2b612c93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:47:28.285791 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-559113d2-8044-42f4-a530-5eba2b612c93 None None] MFA Rules not processed for user `f4c762318a844714b585f80fdfa9ef53`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:47:28.287034 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-559113d2-8044-42f4-a530-5eba2b612c93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:47:28.372630 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1349/2703] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:47:28 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:47:28.377030 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3a2c57-8545-48f6-8509-ea0ea220e7ff admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:47:28.378530 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-8b3a2c57-8545-48f6-8509-ea0ea220e7ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:47:28.455480 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3a2c57-8545-48f6-8509-ea0ea220e7ff admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:47:28.461089 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ff55a2-143d-42df-94ab-d231b957d736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:47:28.462195 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ff55a2-143d-42df-94ab-d231b957d736 admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:47:28.462888 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ff55a2-143d-42df-94ab-d231b957d736 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:47:28.463376 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ff55a2-143d-42df-94ab-d231b957d736 admin admin] PATH_INFO: `/v3/projects` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:47:28.464226 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ff55a2-143d-42df-94ab-d231b957d736 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:47:28.468081 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ff55a2-143d-42df-94ab-d231b957d736 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:47:28.477729 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:47:28.567100 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1355/2704] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:47:28 2022] POST /identity/v3/projects => generated 383 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:47:28.571300 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559113d2-8044-42f4-a530-5eba2b612c93 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:47:28.581952 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559113d2-8044-42f4-a530-5eba2b612c93 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:47:28.656791 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbdab47-ca1f-481b-b2c3-9a03983a985e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:47:28.657602 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbdab47-ca1f-481b-b2c3-9a03983a985e admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:47:28.657892 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbdab47-ca1f-481b-b2c3-9a03983a985e admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:47:28.658163 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbdab47-ca1f-481b-b2c3-9a03983a985e admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:47:28.658796 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbdab47-ca1f-481b-b2c3-9a03983a985e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:47:28.660877 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbdab47-ca1f-481b-b2c3-9a03983a985e admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:47:28.755585 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1350/2705] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:47:28 2022] POST /identity/v3/users => generated 298 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:47:28.760454 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ff55a2-143d-42df-94ab-d231b957d736 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:47:28.770896 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ff55a2-143d-42df-94ab-d231b957d736 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:47:28.775863 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5cc4d1-0ae4-4646-b5a1-d71563a92588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:47:28.776554 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5cc4d1-0ae4-4646-b5a1-d71563a92588 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:47:28.776903 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5cc4d1-0ae4-4646-b5a1-d71563a92588 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:47:28.777245 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5cc4d1-0ae4-4646-b5a1-d71563a92588 admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:47:28.778067 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5cc4d1-0ae4-4646-b5a1-d71563a92588 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:47:28.779874 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5cc4d1-0ae4-4646-b5a1-d71563a92588 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:47:28.791438 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1356/2706] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:47:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:47:28.795137 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbdab47-ca1f-481b-b2c3-9a03983a985e admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:47:28.802385 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:47:28.806991 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbdab47-ca1f-481b-b2c3-9a03983a985e admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:47:28.812227 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc115b4-bb79-489b-8adb-1de9a4c6720d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:47:28.814187 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc115b4-bb79-489b-8adb-1de9a4c6720d admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:47:28.814187 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc115b4-bb79-489b-8adb-1de9a4c6720d admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:47:28.814187 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc115b4-bb79-489b-8adb-1de9a4c6720d admin admin] PATH_INFO: `/v3/projects/087c6e23abf04c4eb97e99b470083437/users/45f5bd813d554285b460c929fa11e128/roles/6bbb643e0d8d4bc0a9b27c26edf59216` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:47:28.832725 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc115b4-bb79-489b-8adb-1de9a4c6720d admin admin] RBAC: Authorizing `identity:create_grant(project_id=087c6e23abf04c4eb97e99b470083437, user_id=45f5bd813d554285b460c929fa11e128, role_id=6bbb643e0d8d4bc0a9b27c26edf59216)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:47:28.835971 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc115b4-bb79-489b-8adb-1de9a4c6720d admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:47:28.954617 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1351/2707] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:47:28 2022] PUT /identity/v3/projects/087c6e23abf04c4eb97e99b470083437/users/45f5bd813d554285b460c929fa11e128/roles/6bbb643e0d8d4bc0a9b27c26edf59216 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 07:47:28.958577 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5cc4d1-0ae4-4646-b5a1-d71563a92588 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:47:28.986498 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5cc4d1-0ae4-4646-b5a1-d71563a92588 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:47:29.053383 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7270302e-b205-4abb-a1e8-58571c85f4cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:47:29.054026 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7270302e-b205-4abb-a1e8-58571c85f4cf admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:47:29.054306 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7270302e-b205-4abb-a1e8-58571c85f4cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:47:29.054574 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7270302e-b205-4abb-a1e8-58571c85f4cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:47:29.055408 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7270302e-b205-4abb-a1e8-58571c85f4cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:47:29.057034 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7270302e-b205-4abb-a1e8-58571c85f4cf admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:47:29.067851 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1357/2708] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:47:28 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:47:29.072522 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc115b4-bb79-489b-8adb-1de9a4c6720d admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:47:29.082874 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc115b4-bb79-489b-8adb-1de9a4c6720d admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:47:29.089857 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f4050e-6bd8-4c03-90c3-9f0b78de34da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:47:29.091025 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f4050e-6bd8-4c03-90c3-9f0b78de34da admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:47:29.091515 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f4050e-6bd8-4c03-90c3-9f0b78de34da admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:47:29.095535 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f4050e-6bd8-4c03-90c3-9f0b78de34da admin admin] PATH_INFO: `/v3/projects/087c6e23abf04c4eb97e99b470083437/users/45f5bd813d554285b460c929fa11e128/roles/6bbb643e0d8d4bc0a9b27c26edf59216` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:47:29.099795 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f4050e-6bd8-4c03-90c3-9f0b78de34da admin admin] RBAC: Authorizing `identity:create_grant(project_id=087c6e23abf04c4eb97e99b470083437, user_id=45f5bd813d554285b460c929fa11e128, role_id=6bbb643e0d8d4bc0a9b27c26edf59216)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:47:29.100186 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f4050e-6bd8-4c03-90c3-9f0b78de34da admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:47:29.116174 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1352/2709] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:47:29 2022] PUT /identity/v3/projects/087c6e23abf04c4eb97e99b470083437/users/45f5bd813d554285b460c929fa11e128/roles/6bbb643e0d8d4bc0a9b27c26edf59216 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:47:35.621444 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab89db8-b702-448f-88b1-91e1e185b328 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:47:35.621444 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab89db8-b702-448f-88b1-91e1e185b328 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:47:35.621444 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab89db8-b702-448f-88b1-91e1e185b328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:47:35.654408 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-8ab89db8-b702-448f-88b1-91e1e185b328 None None] MFA Rules not processed for user `45f5bd813d554285b460c929fa11e128`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:47:35.655672 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-8ab89db8-b702-448f-88b1-91e1e185b328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:47:35.731313 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1358/2710] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:47:35 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 07:47:35.744234 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f4050e-6bd8-4c03-90c3-9f0b78de34da admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:47:35.746019 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-b7f4050e-6bd8-4c03-90c3-9f0b78de34da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:47:35.841011 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f4050e-6bd8-4c03-90c3-9f0b78de34da admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:47:35.845190 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19279379-f0ea-4a11-8c40-5edad4b0b32a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37438b032a1f46d3bdf7121ad0cffbc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:47:35.845919 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19279379-f0ea-4a11-8c40-5edad4b0b32a service glance] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:47:35.846269 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19279379-f0ea-4a11-8c40-5edad4b0b32a service glance] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:47:35.846608 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19279379-f0ea-4a11-8c40-5edad4b0b32a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:47:35.908590 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19279379-f0ea-4a11-8c40-5edad4b0b32a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:47:35.910197 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19279379-f0ea-4a11-8c40-5edad4b0b32a service glance] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:47:35.921509 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1353/2711] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:47:35 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 07:47:36.221788 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3997abaa-7769-4d00-b85f-0b29bc170b53 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:47:36.222080 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3997abaa-7769-4d00-b85f-0b29bc170b53 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:47:36.222366 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3997abaa-7769-4d00-b85f-0b29bc170b53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:47:36.277167 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-3997abaa-7769-4d00-b85f-0b29bc170b53 None None] MFA Rules not processed for user `f9c68c4bd9074d8d9b893755e28c0807`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:47:36.278681 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-3997abaa-7769-4d00-b85f-0b29bc170b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:47:36.394769 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:47:36.437486 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1359/2712] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Sep 15 07:47:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:47:36.597616 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19279379-f0ea-4a11-8c40-5edad4b0b32a service glance] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:47:36.597616 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-19279379-f0ea-4a11-8c40-5edad4b0b32a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:47:36.664610 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19279379-f0ea-4a11-8c40-5edad4b0b32a service glance] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:47:36.669763 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce00c64b-8ebe-4a15-8af0-c57f43e84ce1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72b90d35087e4d7a80e9417082a1541f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:47:36.672010 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce00c64b-8ebe-4a15-8af0-c57f43e84ce1 service swift] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:47:36.672010 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce00c64b-8ebe-4a15-8af0-c57f43e84ce1 service swift] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:47:36.672010 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce00c64b-8ebe-4a15-8af0-c57f43e84ce1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:47:36.680163 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce00c64b-8ebe-4a15-8af0-c57f43e84ce1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:47:36.681783 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce00c64b-8ebe-4a15-8af0-c57f43e84ce1 service swift] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:47:36.690075 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1354/2713] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Sep 15 07:47:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 07:47:37.922429 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3997abaa-7769-4d00-b85f-0b29bc170b53 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:47:37.923983 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-3997abaa-7769-4d00-b85f-0b29bc170b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:47:38.039220 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3997abaa-7769-4d00-b85f-0b29bc170b53 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:47:38.043956 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c80f783-7958-4575-8530-9885bbc392c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:47:38.046743 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c80f783-7958-4575-8530-9885bbc392c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:47:38.047070 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c80f783-7958-4575-8530-9885bbc392c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:47:38.047322 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c80f783-7958-4575-8530-9885bbc392c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:47:38.049975 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-6c80f783-7958-4575-8530-9885bbc392c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:47:38.148295 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c80f783-7958-4575-8530-9885bbc392c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:47:38.151582 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c80f783-7958-4575-8530-9885bbc392c1 service neutron] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:47:38.166410 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1360/2714] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:47:37 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 07:47:40.721819 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af5b534-b8b9-4761-8e87-38b4864382a9 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:47:40.722760 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af5b534-b8b9-4761-8e87-38b4864382a9 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:47:40.723273 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af5b534-b8b9-4761-8e87-38b4864382a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:47:40.754417 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-2af5b534-b8b9-4761-8e87-38b4864382a9 None None] MFA Rules not processed for user `f9c68c4bd9074d8d9b893755e28c0807`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:47:40.757481 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-2af5b534-b8b9-4761-8e87-38b4864382a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:47:40.776765 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1355/2715] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Sep 15 07:47:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:47:40.825886 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c80f783-7958-4575-8530-9885bbc392c1 service neutron] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:47:40.844863 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c80f783-7958-4575-8530-9885bbc392c1 service neutron] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:47:40.853200 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605c3d49-1eab-451c-801d-4f6ea04bf8d7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72b90d35087e4d7a80e9417082a1541f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:47:40.854057 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605c3d49-1eab-451c-801d-4f6ea04bf8d7 service swift] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:47:40.854445 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605c3d49-1eab-451c-801d-4f6ea04bf8d7 service swift] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:47:40.854813 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605c3d49-1eab-451c-801d-4f6ea04bf8d7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:47:40.873504 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605c3d49-1eab-451c-801d-4f6ea04bf8d7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:47:40.875656 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605c3d49-1eab-451c-801d-4f6ea04bf8d7 service swift] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:47:40.886732 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1361/2716] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Sep 15 07:47:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 07:47:50.073420 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aad7887-ebdd-4d2f-97ab-2e97f8892d91 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:47:50.074019 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aad7887-ebdd-4d2f-97ab-2e97f8892d91 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:47:50.074019 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aad7887-ebdd-4d2f-97ab-2e97f8892d91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:47:50.085541 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:47:50.097819 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-7aad7887-ebdd-4d2f-97ab-2e97f8892d91 None None] MFA Rules not processed for user `f4c762318a844714b585f80fdfa9ef53`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:47:50.099053 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-7aad7887-ebdd-4d2f-97ab-2e97f8892d91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:47:50.113053 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1356/2717] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:47:50 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:47:50.881303 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=62 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:47:55.080052 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f30dee-5f82-4908-952b-596a5c78ef3a None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:47:55.080719 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f30dee-5f82-4908-952b-596a5c78ef3a None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:47:55.080832 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f30dee-5f82-4908-952b-596a5c78ef3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:47:55.111925 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-56f30dee-5f82-4908-952b-596a5c78ef3a None None] MFA Rules not processed for user `f9c68c4bd9074d8d9b893755e28c0807`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:47:55.111925 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-56f30dee-5f82-4908-952b-596a5c78ef3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:47:55.125663 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1362/2718] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Sep 15 07:47:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 07:47:55.137390 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aad7887-ebdd-4d2f-97ab-2e97f8892d91 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:47:55.152861 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aad7887-ebdd-4d2f-97ab-2e97f8892d91 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:47:55.173475 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecc465f-c7c7-442f-8de8-4e29bc8dc655 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72b90d35087e4d7a80e9417082a1541f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:47:55.174269 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecc465f-c7c7-442f-8de8-4e29bc8dc655 service swift] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:47:55.174506 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecc465f-c7c7-442f-8de8-4e29bc8dc655 service swift] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:47:55.174727 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecc465f-c7c7-442f-8de8-4e29bc8dc655 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:47:55.186472 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecc465f-c7c7-442f-8de8-4e29bc8dc655 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:47:55.195168 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecc465f-c7c7-442f-8de8-4e29bc8dc655 service swift] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:47:55.203848 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1357/2719] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Sep 15 07:47:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 07:48:01.409711 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061fdc3e-cd0d-4874-848b-d74aae97dd4b None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:48:01.410303 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061fdc3e-cd0d-4874-848b-d74aae97dd4b None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:48:01.410466 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061fdc3e-cd0d-4874-848b-d74aae97dd4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:48:01.445527 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-061fdc3e-cd0d-4874-848b-d74aae97dd4b None None] MFA Rules not processed for user `f9c68c4bd9074d8d9b893755e28c0807`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:48:01.445527 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-061fdc3e-cd0d-4874-848b-d74aae97dd4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:48:01.448438 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1363/2720] 10.222.0.42 () {58 vars in 967 bytes} [Thu Sep 15 07:48:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:48:01.460375 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecc465f-c7c7-442f-8de8-4e29bc8dc655 service swift] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:48:01.474473 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecc465f-c7c7-442f-8de8-4e29bc8dc655 service swift] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:48:01.480526 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18559c28-d4de-497f-8029-2b5eaf2ba056 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72b90d35087e4d7a80e9417082a1541f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:48:01.481463 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18559c28-d4de-497f-8029-2b5eaf2ba056 service swift] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:48:01.481933 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18559c28-d4de-497f-8029-2b5eaf2ba056 service swift] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:48:01.482401 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18559c28-d4de-497f-8029-2b5eaf2ba056 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:48:01.493186 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18559c28-d4de-497f-8029-2b5eaf2ba056 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:48:01.495094 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18559c28-d4de-497f-8029-2b5eaf2ba056 service swift] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:48:01.505446 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1358/2721] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Sep 15 07:48:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 07:48:06.534097 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061fdc3e-cd0d-4874-848b-d74aae97dd4b None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:48:06.535610 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-061fdc3e-cd0d-4874-848b-d74aae97dd4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:48:06.602670 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061fdc3e-cd0d-4874-848b-d74aae97dd4b None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:48:06.607193 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb9f77d-e37c-46e4-8dda-af26af82e01c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:48:06.607811 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb9f77d-e37c-46e4-8dda-af26af82e01c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:48:06.608088 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb9f77d-e37c-46e4-8dda-af26af82e01c admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:48:06.608356 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb9f77d-e37c-46e4-8dda-af26af82e01c admin admin] PATH_INFO: `/v3/users/4d66f4fd90e74287ae9891e0364ab2cd` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:48:06.609466 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb9f77d-e37c-46e4-8dda-af26af82e01c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d66f4fd90e74287ae9891e0364ab2cd)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:48:06.611387 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb9f77d-e37c-46e4-8dda-af26af82e01c admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:48:06.776333 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-ecb9f77d-e37c-46e4-8dda-af26af82e01c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d66f4fd90e74287ae9891e0364ab2cd'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:48:06.854085 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-ecb9f77d-e37c-46e4-8dda-af26af82e01c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d66f4fd90e74287ae9891e0364ab2cd'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:48:06.854704 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-ecb9f77d-e37c-46e4-8dda-af26af82e01c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d66f4fd90e74287ae9891e0364ab2cd'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:48:06.855303 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-ecb9f77d-e37c-46e4-8dda-af26af82e01c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d66f4fd90e74287ae9891e0364ab2cd'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:48:06.862658 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-ecb9f77d-e37c-46e4-8dda-af26af82e01c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d66f4fd90e74287ae9891e0364ab2cd'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:48:06.884785 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1364/2722] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:48:06 2022] DELETE /identity/v3/users/4d66f4fd90e74287ae9891e0364ab2cd => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:48:07.144660 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18559c28-d4de-497f-8029-2b5eaf2ba056 service swift] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:48:07.155656 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18559c28-d4de-497f-8029-2b5eaf2ba056 service swift] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:48:07.227597 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755a5c20-03cd-487c-9038-4c3e34f4f970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:48:07.228387 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755a5c20-03cd-487c-9038-4c3e34f4f970 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:48:07.228674 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755a5c20-03cd-487c-9038-4c3e34f4f970 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:48:07.228953 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755a5c20-03cd-487c-9038-4c3e34f4f970 admin admin] PATH_INFO: `/v3/projects/988cf6a6fa2b452e8c4c017f282d144c` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:48:07.230149 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755a5c20-03cd-487c-9038-4c3e34f4f970 admin admin] RBAC: Authorizing `identity:delete_project(project_id=988cf6a6fa2b452e8c4c017f282d144c)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:48:07.232060 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755a5c20-03cd-487c-9038-4c3e34f4f970 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:48:07.371395 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-755a5c20-03cd-487c-9038-4c3e34f4f970 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '988cf6a6fa2b452e8c4c017f282d144c', 'request_id': 'req-755a5c20-03cd-487c-9038-4c3e34f4f970'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:48:07.377468 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-755a5c20-03cd-487c-9038-4c3e34f4f970 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '988cf6a6fa2b452e8c4c017f282d144c', 'request_id': 'req-755a5c20-03cd-487c-9038-4c3e34f4f970'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:48:07.470898 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-755a5c20-03cd-487c-9038-4c3e34f4f970 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '988cf6a6fa2b452e8c4c017f282d144c', 'request_id': 'req-755a5c20-03cd-487c-9038-4c3e34f4f970'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:48:07.482945 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-755a5c20-03cd-487c-9038-4c3e34f4f970 admin admin] The token cache is being invalidate because project 988cf6a6fa2b452e8c4c017f282d144c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:48:07.483235 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-755a5c20-03cd-487c-9038-4c3e34f4f970 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:48:07.483846 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-755a5c20-03cd-487c-9038-4c3e34f4f970 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:48:07.485314 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1359/2723] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:48:07 2022] DELETE /identity/v3/projects/988cf6a6fa2b452e8c4c017f282d144c => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:48:07.490652 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afc6377-1eab-4af4-8759-3eae2e6015b1 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:48:07.490939 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afc6377-1eab-4af4-8759-3eae2e6015b1 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:48:07.491219 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afc6377-1eab-4af4-8759-3eae2e6015b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:48:07.512382 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-8afc6377-1eab-4af4-8759-3eae2e6015b1 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:48:07.513713 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-8afc6377-1eab-4af4-8759-3eae2e6015b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:48:07.571714 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1365/2724] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:48:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:48:07.579286 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008ed212-5574-458d-b31e-511e9c179af9 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:48:07.579574 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008ed212-5574-458d-b31e-511e9c179af9 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:48:07.579851 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008ed212-5574-458d-b31e-511e9c179af9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:48:07.600816 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-008ed212-5574-458d-b31e-511e9c179af9 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:48:07.602050 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-008ed212-5574-458d-b31e-511e9c179af9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:48:07.610094 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1360/2725] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:48:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:48:07.614303 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afc6377-1eab-4af4-8759-3eae2e6015b1 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:48:07.624127 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afc6377-1eab-4af4-8759-3eae2e6015b1 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:48:07.628608 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e9c3a3-f66f-48eb-adea-5967fcafb6cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:48:07.629201 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e9c3a3-f66f-48eb-adea-5967fcafb6cf admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:48:07.629478 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e9c3a3-f66f-48eb-adea-5967fcafb6cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:48:07.630292 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e9c3a3-f66f-48eb-adea-5967fcafb6cf admin admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:48:07.630944 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e9c3a3-f66f-48eb-adea-5967fcafb6cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:48:07.631240 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e9c3a3-f66f-48eb-adea-5967fcafb6cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:48:07.632819 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e9c3a3-f66f-48eb-adea-5967fcafb6cf admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:48:07.634425 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:48:07.645243 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1366/2726] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 15 07:48:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:48:07.650299 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008ed212-5574-458d-b31e-511e9c179af9 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:48:07.659563 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008ed212-5574-458d-b31e-511e9c179af9 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:48:07.664283 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b39a8e-8097-4e62-9e81-7b66323a0be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:48:07.664882 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b39a8e-8097-4e62-9e81-7b66323a0be1 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:48:07.665163 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b39a8e-8097-4e62-9e81-7b66323a0be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:48:07.665433 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b39a8e-8097-4e62-9e81-7b66323a0be1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:48:07.666098 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b39a8e-8097-4e62-9e81-7b66323a0be1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:48:07.667974 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b39a8e-8097-4e62-9e81-7b66323a0be1 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:48:07.759730 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1361/2727] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:48:07 2022] POST /identity/v3/projects => generated 383 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:48:07.763874 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e9c3a3-f66f-48eb-adea-5967fcafb6cf admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:48:07.774830 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e9c3a3-f66f-48eb-adea-5967fcafb6cf admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:48:07.830777 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa1aac7-0b15-4521-b523-74bd366680ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:48:07.831401 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa1aac7-0b15-4521-b523-74bd366680ed admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:48:07.831684 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa1aac7-0b15-4521-b523-74bd366680ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:48:07.831957 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa1aac7-0b15-4521-b523-74bd366680ed admin admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:48:07.832597 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa1aac7-0b15-4521-b523-74bd366680ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:48:07.835142 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa1aac7-0b15-4521-b523-74bd366680ed admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:48:07.968912 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1367/2728] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:48:07 2022] POST /identity/v3/users => generated 292 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:48:07.972989 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b39a8e-8097-4e62-9e81-7b66323a0be1 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:48:07.983328 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b39a8e-8097-4e62-9e81-7b66323a0be1 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:48:07.987814 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35be73d-14e5-4b39-b7aa-fde6bf6cfe2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:48:07.988414 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35be73d-14e5-4b39-b7aa-fde6bf6cfe2d admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:48:07.988700 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35be73d-14e5-4b39-b7aa-fde6bf6cfe2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:48:07.988976 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35be73d-14e5-4b39-b7aa-fde6bf6cfe2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:48:07.990439 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35be73d-14e5-4b39-b7aa-fde6bf6cfe2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:48:07.992045 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35be73d-14e5-4b39-b7aa-fde6bf6cfe2d admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:48:08.001953 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1362/2729] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:48:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:48:08.006577 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa1aac7-0b15-4521-b523-74bd366680ed admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:48:08.016451 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa1aac7-0b15-4521-b523-74bd366680ed admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:48:08.020988 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ec6124-cd79-4259-b5e5-cf323769268a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:48:08.021571 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ec6124-cd79-4259-b5e5-cf323769268a admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:48:08.021875 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ec6124-cd79-4259-b5e5-cf323769268a admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:48:08.022146 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ec6124-cd79-4259-b5e5-cf323769268a admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:48:08.022774 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ec6124-cd79-4259-b5e5-cf323769268a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:48:08.024337 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ec6124-cd79-4259-b5e5-cf323769268a admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:48:08.034665 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1368/2730] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:48:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:48:08.038894 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35be73d-14e5-4b39-b7aa-fde6bf6cfe2d admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:48:08.048701 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35be73d-14e5-4b39-b7aa-fde6bf6cfe2d admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:48:08.053912 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b056554-2e8a-4bf8-8830-8433ac19b676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:48:08.054555 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b056554-2e8a-4bf8-8830-8433ac19b676 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:48:08.054839 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b056554-2e8a-4bf8-8830-8433ac19b676 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:48:08.055117 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b056554-2e8a-4bf8-8830-8433ac19b676 admin admin] PATH_INFO: `/v3/projects/6cfeb3b1078043f9bf5328833d996243/users/d1bad5a4899d459081a69ea89ac540fe/roles/053e12af84e943e699460a9d76c2072b` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:48:08.069841 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b056554-2e8a-4bf8-8830-8433ac19b676 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cfeb3b1078043f9bf5328833d996243, user_id=d1bad5a4899d459081a69ea89ac540fe, role_id=053e12af84e943e699460a9d76c2072b)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:48:08.072475 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b056554-2e8a-4bf8-8830-8433ac19b676 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:48:08.169187 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:48:08.174854 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1363/2731] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:48:08 2022] PUT /identity/v3/projects/6cfeb3b1078043f9bf5328833d996243/users/d1bad5a4899d459081a69ea89ac540fe/roles/053e12af84e943e699460a9d76c2072b => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:48:08.185600 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed6de1e-6b1c-4c0e-b791-a56bd1b19a57 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:48:08.185910 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed6de1e-6b1c-4c0e-b791-a56bd1b19a57 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:48:08.186190 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed6de1e-6b1c-4c0e-b791-a56bd1b19a57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:48:08.214550 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-0ed6de1e-6b1c-4c0e-b791-a56bd1b19a57 None None] MFA Rules not processed for user `d1bad5a4899d459081a69ea89ac540fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:48:08.215847 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-0ed6de1e-6b1c-4c0e-b791-a56bd1b19a57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:48:08.280704 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1369/2732] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:48:08 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:48:08.293055 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b056554-2e8a-4bf8-8830-8433ac19b676 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:48:08.294764 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-7b056554-2e8a-4bf8-8830-8433ac19b676 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:48:08.353782 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b056554-2e8a-4bf8-8830-8433ac19b676 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:48:08.357526 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4073a8-5f91-41a3-a299-6c0090de2b8f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecf6642b7b84d3c9756bb41ab3349d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:48:08.361194 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4073a8-5f91-41a3-a299-6c0090de2b8f service cinder] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:48:08.361194 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4073a8-5f91-41a3-a299-6c0090de2b8f service cinder] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:48:08.361194 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4073a8-5f91-41a3-a299-6c0090de2b8f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:48:08.367586 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4073a8-5f91-41a3-a299-6c0090de2b8f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:48:08.369082 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4073a8-5f91-41a3-a299-6c0090de2b8f service cinder] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:48:08.378207 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1364/2733] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:48:08 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:48:11.369934 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed6de1e-6b1c-4c0e-b791-a56bd1b19a57 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:48:11.431678 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed6de1e-6b1c-4c0e-b791-a56bd1b19a57 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:48:11.436170 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19fbd57-e819-4cf6-857c-9064c45a426f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:48:11.436792 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19fbd57-e819-4cf6-857c-9064c45a426f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:48:11.437069 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19fbd57-e819-4cf6-857c-9064c45a426f admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:48:11.437340 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19fbd57-e819-4cf6-857c-9064c45a426f admin admin] PATH_INFO: `/v3/users/d1bad5a4899d459081a69ea89ac540fe` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:48:11.438478 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19fbd57-e819-4cf6-857c-9064c45a426f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1bad5a4899d459081a69ea89ac540fe)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:48:11.440378 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19fbd57-e819-4cf6-857c-9064c45a426f admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:48:11.443413 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:48:11.592575 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-b19fbd57-e819-4cf6-857c-9064c45a426f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1bad5a4899d459081a69ea89ac540fe'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:48:11.635982 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-b19fbd57-e819-4cf6-857c-9064c45a426f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1bad5a4899d459081a69ea89ac540fe'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:48:11.636555 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-b19fbd57-e819-4cf6-857c-9064c45a426f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1bad5a4899d459081a69ea89ac540fe'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:48:11.637158 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-b19fbd57-e819-4cf6-857c-9064c45a426f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1bad5a4899d459081a69ea89ac540fe'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:48:11.644868 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-b19fbd57-e819-4cf6-857c-9064c45a426f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1bad5a4899d459081a69ea89ac540fe'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:48:11.664288 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1370/2734] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:48:11 2022] DELETE /identity/v3/users/d1bad5a4899d459081a69ea89ac540fe => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:48:11.668761 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8934946-9693-4cc8-baa2-8e6758c65843 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:48:11.669233 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8934946-9693-4cc8-baa2-8e6758c65843 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:48:11.669332 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8934946-9693-4cc8-baa2-8e6758c65843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:48:11.688627 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-b8934946-9693-4cc8-baa2-8e6758c65843 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:48:11.689850 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-b8934946-9693-4cc8-baa2-8e6758c65843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:48:11.752709 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1365/2735] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:48:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:48:11.762872 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19fbd57-e819-4cf6-857c-9064c45a426f admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:48:11.764268 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-b19fbd57-e819-4cf6-857c-9064c45a426f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:48:11.821209 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19fbd57-e819-4cf6-857c-9064c45a426f admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:48:11.824800 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfbe813-5f59-4d70-87c8-87dcaca3832a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:48:11.825400 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfbe813-5f59-4d70-87c8-87dcaca3832a service neutron] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:48:11.825715 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfbe813-5f59-4d70-87c8-87dcaca3832a service neutron] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:48:11.825986 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfbe813-5f59-4d70-87c8-87dcaca3832a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:48:11.835801 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfbe813-5f59-4d70-87c8-87dcaca3832a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:48:11.837215 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfbe813-5f59-4d70-87c8-87dcaca3832a service neutron] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:48:11.846899 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1371/2736] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:48:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:48:12.295416 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8934946-9693-4cc8-baa2-8e6758c65843 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:48:12.296945 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-b8934946-9693-4cc8-baa2-8e6758c65843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:48:12.312386 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8934946-9693-4cc8-baa2-8e6758c65843 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:48:12.317288 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480f81f2-36c2-4c4d-bc63-497a6cd30806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:48:12.317960 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480f81f2-36c2-4c4d-bc63-497a6cd30806 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:48:12.318244 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480f81f2-36c2-4c4d-bc63-497a6cd30806 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:48:12.318781 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480f81f2-36c2-4c4d-bc63-497a6cd30806 admin admin] PATH_INFO: `/v3/projects/6cfeb3b1078043f9bf5328833d996243` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:48:12.319677 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480f81f2-36c2-4c4d-bc63-497a6cd30806 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6cfeb3b1078043f9bf5328833d996243)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:48:12.321790 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480f81f2-36c2-4c4d-bc63-497a6cd30806 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:48:12.426936 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-480f81f2-36c2-4c4d-bc63-497a6cd30806 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6cfeb3b1078043f9bf5328833d996243', 'request_id': 'req-480f81f2-36c2-4c4d-bc63-497a6cd30806'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:48:12.434909 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-480f81f2-36c2-4c4d-bc63-497a6cd30806 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6cfeb3b1078043f9bf5328833d996243', 'request_id': 'req-480f81f2-36c2-4c4d-bc63-497a6cd30806'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:48:12.517974 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-480f81f2-36c2-4c4d-bc63-497a6cd30806 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6cfeb3b1078043f9bf5328833d996243', 'request_id': 'req-480f81f2-36c2-4c4d-bc63-497a6cd30806'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:48:12.527783 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-480f81f2-36c2-4c4d-bc63-497a6cd30806 admin admin] The token cache is being invalidate because project 6cfeb3b1078043f9bf5328833d996243 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:48:12.528181 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-480f81f2-36c2-4c4d-bc63-497a6cd30806 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:48:12.529042 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-480f81f2-36c2-4c4d-bc63-497a6cd30806 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:48:12.531144 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1366/2737] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:48:12 2022] DELETE /identity/v3/projects/6cfeb3b1078043f9bf5328833d996243 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:48:21.842609 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=51,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:48:22.524387 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=81,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:48:26.502989 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfbe813-5f59-4d70-87c8-87dcaca3832a service neutron] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:48:26.504391 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-dbfbe813-5f59-4d70-87c8-87dcaca3832a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:48:26.516340 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfbe813-5f59-4d70-87c8-87dcaca3832a service neutron] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:48:26.564410 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2675105-b6fd-4a38-b984-793e5e579058 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:48:26.565005 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2675105-b6fd-4a38-b984-793e5e579058 service neutron] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:48:26.565284 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2675105-b6fd-4a38-b984-793e5e579058 service neutron] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:48:26.565557 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2675105-b6fd-4a38-b984-793e5e579058 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:48:26.567284 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-c2675105-b6fd-4a38-b984-793e5e579058 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:48:26.629089 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2675105-b6fd-4a38-b984-793e5e579058 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:48:26.630532 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2675105-b6fd-4a38-b984-793e5e579058 service neutron] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:48:26.640692 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1372/2738] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:48:26 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:48:36.636501 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=50 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:48:49.519591 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480f81f2-36c2-4c4d-bc63-497a6cd30806 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:48:49.521087 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-480f81f2-36c2-4c4d-bc63-497a6cd30806 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:48:49.534116 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480f81f2-36c2-4c4d-bc63-497a6cd30806 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:48:49.537744 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695eb3c6-020e-48c4-a64d-97225cad2c77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:48:49.538359 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695eb3c6-020e-48c4-a64d-97225cad2c77 service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:48:49.538641 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695eb3c6-020e-48c4-a64d-97225cad2c77 service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:48:49.538922 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695eb3c6-020e-48c4-a64d-97225cad2c77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:48:49.540642 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-695eb3c6-020e-48c4-a64d-97225cad2c77 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:48:49.643999 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695eb3c6-020e-48c4-a64d-97225cad2c77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:48:49.645345 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695eb3c6-020e-48c4-a64d-97225cad2c77 service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:48:49.666804 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1367/2739] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:48:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:48:50.875435 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2675105-b6fd-4a38-b984-793e5e579058 service neutron] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:48:50.884348 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2675105-b6fd-4a38-b984-793e5e579058 service neutron] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:48:50.889778 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a4d90f-3c12-432e-87c8-0562dec920f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:48:50.890533 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a4d90f-3c12-432e-87c8-0562dec920f1 service neutron] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:48:50.890820 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a4d90f-3c12-432e-87c8-0562dec920f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:48:50.891097 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a4d90f-3c12-432e-87c8-0562dec920f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:48:50.892876 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-99a4d90f-3c12-432e-87c8-0562dec920f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:48:50.906889 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a4d90f-3c12-432e-87c8-0562dec920f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:48:50.908317 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a4d90f-3c12-432e-87c8-0562dec920f1 service neutron] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:48:50.920615 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1373/2740] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:48:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:48:59.663997 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=43 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:49:00.914512 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=6 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:49:08.284696 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695eb3c6-020e-48c4-a64d-97225cad2c77 service neutron] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:49:08.287574 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-695eb3c6-020e-48c4-a64d-97225cad2c77 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:49:08.301286 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695eb3c6-020e-48c4-a64d-97225cad2c77 service neutron] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:49:08.307143 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cda82a-b834-448c-b17f-3f6411e25778 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a080c7ab9b684abca6828067f2572c3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:49:08.309968 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cda82a-b834-448c-b17f-3f6411e25778 service nova] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:49:08.309968 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cda82a-b834-448c-b17f-3f6411e25778 service nova] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:49:08.309968 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cda82a-b834-448c-b17f-3f6411e25778 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:49:08.310293 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-22cda82a-b834-448c-b17f-3f6411e25778 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:49:08.322922 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cda82a-b834-448c-b17f-3f6411e25778 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:49:08.324500 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cda82a-b834-448c-b17f-3f6411e25778 service nova] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:49:08.335737 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1368/2741] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:49:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:49:13.353303 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf813e2-5e58-493d-ab86-541efa8d310c None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:49:13.353842 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf813e2-5e58-493d-ab86-541efa8d310c None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:49:13.353968 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf813e2-5e58-493d-ab86-541efa8d310c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:49:13.372938 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-adf813e2-5e58-493d-ab86-541efa8d310c None None] MFA Rules not processed for user `f8022d5f904e4ae294b2cdf21aa11212`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:49:13.374177 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-adf813e2-5e58-493d-ab86-541efa8d310c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:49:13.430526 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1374/2742] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:49:13 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:49:14.137101 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cda82a-b834-448c-b17f-3f6411e25778 service nova] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:49:14.138606 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-22cda82a-b834-448c-b17f-3f6411e25778 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:49:14.210526 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cda82a-b834-448c-b17f-3f6411e25778 service nova] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:49:14.215911 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4dba99-6b5a-4267-8f0d-7c77adc54f7e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd87846b9a134248b28e0521b0a880c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:49:14.216515 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4dba99-6b5a-4267-8f0d-7c77adc54f7e service placement] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:49:14.216798 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4dba99-6b5a-4267-8f0d-7c77adc54f7e service placement] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:49:14.217079 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4dba99-6b5a-4267-8f0d-7c77adc54f7e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:49:14.218820 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-cd4dba99-6b5a-4267-8f0d-7c77adc54f7e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:49:14.230778 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4dba99-6b5a-4267-8f0d-7c77adc54f7e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:49:14.232647 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4dba99-6b5a-4267-8f0d-7c77adc54f7e service placement] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:49:14.242189 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1369/2743] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:49:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 07:49:17.179475 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf813e2-5e58-493d-ab86-541efa8d310c None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:49:17.197125 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf813e2-5e58-493d-ab86-541efa8d310c None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:49:17.197125 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a0665a-2c4f-452c-b427-97bedc0c8c5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:49:17.197125 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a0665a-2c4f-452c-b427-97bedc0c8c5c service neutron] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:49:17.197125 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a0665a-2c4f-452c-b427-97bedc0c8c5c service neutron] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:49:17.197493 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a0665a-2c4f-452c-b427-97bedc0c8c5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:49:17.197493 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-18a0665a-2c4f-452c-b427-97bedc0c8c5c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:49:17.206232 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a0665a-2c4f-452c-b427-97bedc0c8c5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:49:17.207644 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a0665a-2c4f-452c-b427-97bedc0c8c5c service neutron] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:49:17.215742 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1375/2744] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:49:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:49:20.107296 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4dba99-6b5a-4267-8f0d-7c77adc54f7e service placement] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:49:20.108702 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-cd4dba99-6b5a-4267-8f0d-7c77adc54f7e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:49:20.125610 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4dba99-6b5a-4267-8f0d-7c77adc54f7e service placement] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:49:20.131340 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5607aab-b0d4-41de-a74e-aa0de8976380 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a080c7ab9b684abca6828067f2572c3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:49:20.131951 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5607aab-b0d4-41de-a74e-aa0de8976380 service nova] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:49:20.132229 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5607aab-b0d4-41de-a74e-aa0de8976380 service nova] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:49:20.133306 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5607aab-b0d4-41de-a74e-aa0de8976380 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:49:20.135907 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-c5607aab-b0d4-41de-a74e-aa0de8976380 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:49:20.148550 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5607aab-b0d4-41de-a74e-aa0de8976380 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:49:20.150293 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5607aab-b0d4-41de-a74e-aa0de8976380 service nova] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:49:20.162462 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1370/2745] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:49:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:49:24.734532 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a0665a-2c4f-452c-b427-97bedc0c8c5c service neutron] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:49:24.748483 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a0665a-2c4f-452c-b427-97bedc0c8c5c service neutron] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:49:24.754818 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8bd8e5-afbf-4469-9cbf-677bad6dcf1b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a080c7ab9b684abca6828067f2572c3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:49:24.755407 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8bd8e5-afbf-4469-9cbf-677bad6dcf1b service nova] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:49:24.755683 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8bd8e5-afbf-4469-9cbf-677bad6dcf1b service nova] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:49:24.755954 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8bd8e5-afbf-4469-9cbf-677bad6dcf1b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:49:24.758088 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-3b8bd8e5-afbf-4469-9cbf-677bad6dcf1b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:49:24.770082 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8bd8e5-afbf-4469-9cbf-677bad6dcf1b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:49:24.771642 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8bd8e5-afbf-4469-9cbf-677bad6dcf1b service nova] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:49:24.782141 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1376/2746] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:49:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:49:30.156670 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=44 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:49:33.959512 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5607aab-b0d4-41de-a74e-aa0de8976380 service nova] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:49:33.969850 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5607aab-b0d4-41de-a74e-aa0de8976380 service nova] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:49:33.973409 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2802134-d67d-4bca-bcf2-267ab6e54223 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:49:33.974028 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2802134-d67d-4bca-bcf2-267ab6e54223 service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:49:33.974312 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2802134-d67d-4bca-bcf2-267ab6e54223 service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:49:33.974612 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2802134-d67d-4bca-bcf2-267ab6e54223 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:49:33.976915 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-c2802134-d67d-4bca-bcf2-267ab6e54223 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:49:34.040932 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2802134-d67d-4bca-bcf2-267ab6e54223 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:49:34.042388 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2802134-d67d-4bca-bcf2-267ab6e54223 service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:49:34.052065 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1371/2747] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:49:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:49:34.777746 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=37 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:49:42.694397 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8bd8e5-afbf-4469-9cbf-677bad6dcf1b service nova] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:49:42.696589 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-3b8bd8e5-afbf-4469-9cbf-677bad6dcf1b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:49:42.710447 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8bd8e5-afbf-4469-9cbf-677bad6dcf1b service nova] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:49:42.715227 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36892c7f-8a5b-4000-8af7-033d7bea3927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:49:42.715879 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36892c7f-8a5b-4000-8af7-033d7bea3927 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:49:42.716302 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36892c7f-8a5b-4000-8af7-033d7bea3927 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:49:42.716576 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36892c7f-8a5b-4000-8af7-033d7bea3927 admin admin] PATH_INFO: `/v3/users/ebd85e42855448268c89a200f4392316` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:49:42.717702 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36892c7f-8a5b-4000-8af7-033d7bea3927 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebd85e42855448268c89a200f4392316)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:49:42.719594 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36892c7f-8a5b-4000-8af7-033d7bea3927 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:49:42.896336 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-36892c7f-8a5b-4000-8af7-033d7bea3927 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebd85e42855448268c89a200f4392316'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:49:42.965881 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-36892c7f-8a5b-4000-8af7-033d7bea3927 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebd85e42855448268c89a200f4392316'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:49:42.966502 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-36892c7f-8a5b-4000-8af7-033d7bea3927 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebd85e42855448268c89a200f4392316'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:49:42.968263 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-36892c7f-8a5b-4000-8af7-033d7bea3927 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebd85e42855448268c89a200f4392316'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:49:42.974746 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-36892c7f-8a5b-4000-8af7-033d7bea3927 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebd85e42855448268c89a200f4392316'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:49:42.992720 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1377/2748] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:49:42 2022] DELETE /identity/v3/users/ebd85e42855448268c89a200f4392316 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:49:43.360361 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2802134-d67d-4bca-bcf2-267ab6e54223 service neutron] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:49:43.371041 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2802134-d67d-4bca-bcf2-267ab6e54223 service neutron] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:49:43.431927 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5132df-9d4c-433a-a2e4-201ab9c13186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:49:43.432463 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5132df-9d4c-433a-a2e4-201ab9c13186 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:49:43.432939 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5132df-9d4c-433a-a2e4-201ab9c13186 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:49:43.433211 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5132df-9d4c-433a-a2e4-201ab9c13186 admin admin] PATH_INFO: `/v3/projects/af1e008c4bfa41e18488c63ea0f17b27` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:49:43.434403 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5132df-9d4c-433a-a2e4-201ab9c13186 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af1e008c4bfa41e18488c63ea0f17b27)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:49:43.436304 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5132df-9d4c-433a-a2e4-201ab9c13186 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:49:43.544321 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-fe5132df-9d4c-433a-a2e4-201ab9c13186 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af1e008c4bfa41e18488c63ea0f17b27', 'request_id': 'req-fe5132df-9d4c-433a-a2e4-201ab9c13186'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:49:43.550619 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-fe5132df-9d4c-433a-a2e4-201ab9c13186 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af1e008c4bfa41e18488c63ea0f17b27', 'request_id': 'req-fe5132df-9d4c-433a-a2e4-201ab9c13186'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:49:43.640675 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-fe5132df-9d4c-433a-a2e4-201ab9c13186 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af1e008c4bfa41e18488c63ea0f17b27', 'request_id': 'req-fe5132df-9d4c-433a-a2e4-201ab9c13186'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:49:43.656259 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-fe5132df-9d4c-433a-a2e4-201ab9c13186 admin admin] The token cache is being invalidate because project af1e008c4bfa41e18488c63ea0f17b27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:49:43.657200 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-fe5132df-9d4c-433a-a2e4-201ab9c13186 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:49:43.657989 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-fe5132df-9d4c-433a-a2e4-201ab9c13186 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:49:43.661448 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1372/2749] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:49:43 2022] DELETE /identity/v3/projects/af1e008c4bfa41e18488c63ea0f17b27 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:49:43.665051 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36892c7f-8a5b-4000-8af7-033d7bea3927 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:49:43.667674 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-36892c7f-8a5b-4000-8af7-033d7bea3927 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:49:43.699611 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36892c7f-8a5b-4000-8af7-033d7bea3927 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:49:43.764229 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9bcaa7-ea73-481e-8292-1d00f54b6b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:49:43.766327 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9bcaa7-ea73-481e-8292-1d00f54b6b1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:49:43.766327 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9bcaa7-ea73-481e-8292-1d00f54b6b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:49:43.766327 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9bcaa7-ea73-481e-8292-1d00f54b6b1b admin admin] PATH_INFO: `/v3/users/1fb4689f1f2e48b2a52a1d05f1e98b30` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:49:43.766648 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9bcaa7-ea73-481e-8292-1d00f54b6b1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1fb4689f1f2e48b2a52a1d05f1e98b30)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:49:43.768490 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9bcaa7-ea73-481e-8292-1d00f54b6b1b admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:49:43.983479 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-dd9bcaa7-ea73-481e-8292-1d00f54b6b1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1fb4689f1f2e48b2a52a1d05f1e98b30'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:49:44.080426 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-dd9bcaa7-ea73-481e-8292-1d00f54b6b1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1fb4689f1f2e48b2a52a1d05f1e98b30'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:49:44.082256 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-dd9bcaa7-ea73-481e-8292-1d00f54b6b1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1fb4689f1f2e48b2a52a1d05f1e98b30'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:49:44.083303 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-dd9bcaa7-ea73-481e-8292-1d00f54b6b1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1fb4689f1f2e48b2a52a1d05f1e98b30'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:49:44.101394 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-dd9bcaa7-ea73-481e-8292-1d00f54b6b1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1fb4689f1f2e48b2a52a1d05f1e98b30'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:49:44.116351 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:49:44.124117 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1378/2750] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:49:43 2022] DELETE /identity/v3/users/1fb4689f1f2e48b2a52a1d05f1e98b30 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:49:44.375003 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5132df-9d4c-433a-a2e4-201ab9c13186 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:49:44.376510 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-fe5132df-9d4c-433a-a2e4-201ab9c13186 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:49:44.389955 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5132df-9d4c-433a-a2e4-201ab9c13186 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:49:44.411263 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:49:44.448005 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a078d267-cea5-4d2b-b814-b00cfdd8a178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:49:44.448632 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a078d267-cea5-4d2b-b814-b00cfdd8a178 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:49:44.448912 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a078d267-cea5-4d2b-b814-b00cfdd8a178 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:49:44.449182 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a078d267-cea5-4d2b-b814-b00cfdd8a178 admin admin] PATH_INFO: `/v3/projects/e5ef5d2e9c5d4352ad39a7d1b1a0170e` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:49:44.450349 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a078d267-cea5-4d2b-b814-b00cfdd8a178 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5ef5d2e9c5d4352ad39a7d1b1a0170e)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:49:44.452355 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a078d267-cea5-4d2b-b814-b00cfdd8a178 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:49:44.597759 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-a078d267-cea5-4d2b-b814-b00cfdd8a178 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5ef5d2e9c5d4352ad39a7d1b1a0170e', 'request_id': 'req-a078d267-cea5-4d2b-b814-b00cfdd8a178'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:49:44.604568 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-a078d267-cea5-4d2b-b814-b00cfdd8a178 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5ef5d2e9c5d4352ad39a7d1b1a0170e', 'request_id': 'req-a078d267-cea5-4d2b-b814-b00cfdd8a178'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:49:44.694177 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-a078d267-cea5-4d2b-b814-b00cfdd8a178 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5ef5d2e9c5d4352ad39a7d1b1a0170e', 'request_id': 'req-a078d267-cea5-4d2b-b814-b00cfdd8a178'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:49:44.706280 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-a078d267-cea5-4d2b-b814-b00cfdd8a178 admin admin] The token cache is being invalidate because project e5ef5d2e9c5d4352ad39a7d1b1a0170e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:49:44.706510 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-a078d267-cea5-4d2b-b814-b00cfdd8a178 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:49:44.707158 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-a078d267-cea5-4d2b-b814-b00cfdd8a178 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:49:44.708804 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1373/2751] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:49:44 2022] DELETE /identity/v3/projects/e5ef5d2e9c5d4352ad39a7d1b1a0170e => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:49:46.063359 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981d2a43-4c26-4f9e-820c-43ebc2a9b935 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:49:46.063830 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981d2a43-4c26-4f9e-820c-43ebc2a9b935 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:49:46.063943 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981d2a43-4c26-4f9e-820c-43ebc2a9b935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:49:46.112804 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-981d2a43-4c26-4f9e-820c-43ebc2a9b935 None None] MFA Rules not processed for user `f9c68c4bd9074d8d9b893755e28c0807`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:49:46.114031 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-981d2a43-4c26-4f9e-820c-43ebc2a9b935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:49:46.177658 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1379/2752] 10.222.0.42 () {58 vars in 967 bytes} [Thu Sep 15 07:49:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 07:49:46.320633 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a078d267-cea5-4d2b-b814-b00cfdd8a178 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:49:46.322075 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-a078d267-cea5-4d2b-b814-b00cfdd8a178 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:49:46.411109 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a078d267-cea5-4d2b-b814-b00cfdd8a178 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:49:46.416780 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b476606-383a-411a-b1a4-376eb6907baa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72b90d35087e4d7a80e9417082a1541f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:49:46.417392 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b476606-383a-411a-b1a4-376eb6907baa service swift] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:49:46.417710 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b476606-383a-411a-b1a4-376eb6907baa service swift] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:49:46.417992 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b476606-383a-411a-b1a4-376eb6907baa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:49:46.426159 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b476606-383a-411a-b1a4-376eb6907baa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:49:46.427784 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b476606-383a-411a-b1a4-376eb6907baa service swift] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:49:46.435761 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1374/2753] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Sep 15 07:49:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 15 07:49:54.820235 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981d2a43-4c26-4f9e-820c-43ebc2a9b935 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:49:54.821754 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-981d2a43-4c26-4f9e-820c-43ebc2a9b935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:49:54.889048 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981d2a43-4c26-4f9e-820c-43ebc2a9b935 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:49:54.893887 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb42485-b11a-4f3c-bf31-a21cf0d60273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:49:54.894518 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb42485-b11a-4f3c-bf31-a21cf0d60273 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:49:54.894802 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb42485-b11a-4f3c-bf31-a21cf0d60273 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:49:54.895080 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb42485-b11a-4f3c-bf31-a21cf0d60273 admin admin] PATH_INFO: `/v3/users/f4c762318a844714b585f80fdfa9ef53` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:49:54.896786 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb42485-b11a-4f3c-bf31-a21cf0d60273 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4c762318a844714b585f80fdfa9ef53)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:49:54.898836 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb42485-b11a-4f3c-bf31-a21cf0d60273 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:49:55.043693 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-7fb42485-b11a-4f3c-bf31-a21cf0d60273 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4c762318a844714b585f80fdfa9ef53'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:49:55.125049 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-7fb42485-b11a-4f3c-bf31-a21cf0d60273 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4c762318a844714b585f80fdfa9ef53'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:49:55.125725 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-7fb42485-b11a-4f3c-bf31-a21cf0d60273 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4c762318a844714b585f80fdfa9ef53'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:49:55.126338 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-7fb42485-b11a-4f3c-bf31-a21cf0d60273 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4c762318a844714b585f80fdfa9ef53'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:49:55.134788 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-7fb42485-b11a-4f3c-bf31-a21cf0d60273 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4c762318a844714b585f80fdfa9ef53'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:49:55.136681 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:49:55.152412 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1380/2754] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:49:54 2022] DELETE /identity/v3/users/f4c762318a844714b585f80fdfa9ef53 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:49:55.379431 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b476606-383a-411a-b1a4-376eb6907baa service swift] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:49:55.389854 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b476606-383a-411a-b1a4-376eb6907baa service swift] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:49:55.412848 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:49:55.449371 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968e758b-d096-43e0-9250-894c253e8e19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:49:55.450036 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968e758b-d096-43e0-9250-894c253e8e19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:49:55.450316 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968e758b-d096-43e0-9250-894c253e8e19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:49:55.450589 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968e758b-d096-43e0-9250-894c253e8e19 admin admin] PATH_INFO: `/v3/projects/00091272c5a745f2bf80723a69fbfc61` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:49:55.451691 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968e758b-d096-43e0-9250-894c253e8e19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00091272c5a745f2bf80723a69fbfc61)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:49:55.453616 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968e758b-d096-43e0-9250-894c253e8e19 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:49:55.565198 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-968e758b-d096-43e0-9250-894c253e8e19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00091272c5a745f2bf80723a69fbfc61', 'request_id': 'req-968e758b-d096-43e0-9250-894c253e8e19'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:49:55.572037 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-968e758b-d096-43e0-9250-894c253e8e19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00091272c5a745f2bf80723a69fbfc61', 'request_id': 'req-968e758b-d096-43e0-9250-894c253e8e19'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:49:55.639118 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-968e758b-d096-43e0-9250-894c253e8e19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00091272c5a745f2bf80723a69fbfc61', 'request_id': 'req-968e758b-d096-43e0-9250-894c253e8e19'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:49:55.650357 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-968e758b-d096-43e0-9250-894c253e8e19 admin admin] The token cache is being invalidate because project 00091272c5a745f2bf80723a69fbfc61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:49:55.650931 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-968e758b-d096-43e0-9250-894c253e8e19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:49:55.651554 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-968e758b-d096-43e0-9250-894c253e8e19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:49:55.653080 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1375/2755] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:49:55 2022] DELETE /identity/v3/projects/00091272c5a745f2bf80723a69fbfc61 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:49:55.657214 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb42485-b11a-4f3c-bf31-a21cf0d60273 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:49:55.658664 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-7fb42485-b11a-4f3c-bf31-a21cf0d60273 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:49:55.671984 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb42485-b11a-4f3c-bf31-a21cf0d60273 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:49:55.728708 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac4f511-87f0-4b98-9fc5-e42b969e0a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:49:55.729325 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac4f511-87f0-4b98-9fc5-e42b969e0a4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:49:55.729623 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac4f511-87f0-4b98-9fc5-e42b969e0a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:49:55.729907 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac4f511-87f0-4b98-9fc5-e42b969e0a4c admin admin] PATH_INFO: `/v3/users/45f5bd813d554285b460c929fa11e128` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:49:55.731019 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac4f511-87f0-4b98-9fc5-e42b969e0a4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=45f5bd813d554285b460c929fa11e128)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:49:55.732904 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac4f511-87f0-4b98-9fc5-e42b969e0a4c admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:49:55.946905 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-dac4f511-87f0-4b98-9fc5-e42b969e0a4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45f5bd813d554285b460c929fa11e128'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:49:56.024508 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-dac4f511-87f0-4b98-9fc5-e42b969e0a4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45f5bd813d554285b460c929fa11e128'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:49:56.025160 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-dac4f511-87f0-4b98-9fc5-e42b969e0a4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45f5bd813d554285b460c929fa11e128'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:49:56.025810 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-dac4f511-87f0-4b98-9fc5-e42b969e0a4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45f5bd813d554285b460c929fa11e128'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:49:56.034058 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-dac4f511-87f0-4b98-9fc5-e42b969e0a4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45f5bd813d554285b460c929fa11e128'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:49:56.053955 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1381/2756] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:49:55 2022] DELETE /identity/v3/users/45f5bd813d554285b460c929fa11e128 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:49:56.300661 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968e758b-d096-43e0-9250-894c253e8e19 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:49:56.304231 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-968e758b-d096-43e0-9250-894c253e8e19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:49:56.319308 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968e758b-d096-43e0-9250-894c253e8e19 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:49:56.383233 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c4ff20-0c4b-4077-980c-77f5e0f97955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:49:56.383911 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c4ff20-0c4b-4077-980c-77f5e0f97955 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:49:56.384125 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c4ff20-0c4b-4077-980c-77f5e0f97955 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:49:56.384395 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c4ff20-0c4b-4077-980c-77f5e0f97955 admin admin] PATH_INFO: `/v3/projects/087c6e23abf04c4eb97e99b470083437` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:49:56.385506 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c4ff20-0c4b-4077-980c-77f5e0f97955 admin admin] RBAC: Authorizing `identity:delete_project(project_id=087c6e23abf04c4eb97e99b470083437)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:49:56.387429 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c4ff20-0c4b-4077-980c-77f5e0f97955 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:49:56.493341 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-34c4ff20-0c4b-4077-980c-77f5e0f97955 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '087c6e23abf04c4eb97e99b470083437', 'request_id': 'req-34c4ff20-0c4b-4077-980c-77f5e0f97955'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:49:56.499579 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-34c4ff20-0c4b-4077-980c-77f5e0f97955 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '087c6e23abf04c4eb97e99b470083437', 'request_id': 'req-34c4ff20-0c4b-4077-980c-77f5e0f97955'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:49:56.502128 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:49:56.630033 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-34c4ff20-0c4b-4077-980c-77f5e0f97955 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '087c6e23abf04c4eb97e99b470083437', 'request_id': 'req-34c4ff20-0c4b-4077-980c-77f5e0f97955'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:49:56.642085 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-34c4ff20-0c4b-4077-980c-77f5e0f97955 admin admin] The token cache is being invalidate because project 087c6e23abf04c4eb97e99b470083437 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:49:56.642385 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-34c4ff20-0c4b-4077-980c-77f5e0f97955 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:49:56.643003 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-34c4ff20-0c4b-4077-980c-77f5e0f97955 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:49:56.644481 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1376/2757] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:49:56 2022] DELETE /identity/v3/projects/087c6e23abf04c4eb97e99b470083437 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:50:06.049855 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=59,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:50:06.715447 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:DELETE=3,keystone:SELECT=4 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:50:41.535323 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac4f511-87f0-4b98-9fc5-e42b969e0a4c admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:50:41.536978 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-dac4f511-87f0-4b98-9fc5-e42b969e0a4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:50:41.606706 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac4f511-87f0-4b98-9fc5-e42b969e0a4c admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:50:41.665683 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bcf34d-e672-4077-91ea-f2401261f093 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd87846b9a134248b28e0521b0a880c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:50:41.666380 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bcf34d-e672-4077-91ea-f2401261f093 service placement] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:50:41.666656 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bcf34d-e672-4077-91ea-f2401261f093 service placement] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:50:41.667009 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bcf34d-e672-4077-91ea-f2401261f093 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:50:41.669174 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-57bcf34d-e672-4077-91ea-f2401261f093 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:50:41.680874 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c4ff20-0c4b-4077-980c-77f5e0f97955 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:50:41.682425 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-34c4ff20-0c4b-4077-980c-77f5e0f97955 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:50:41.682689 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bcf34d-e672-4077-91ea-f2401261f093 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:50:41.684279 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bcf34d-e672-4077-91ea-f2401261f093 service placement] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:50:41.696255 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1382/2758] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:50:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:50:41.754217 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c4ff20-0c4b-4077-980c-77f5e0f97955 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:50:41.759501 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29333419-951b-45b8-b591-6e17bf85705d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:50:41.760158 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29333419-951b-45b8-b591-6e17bf85705d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:50:41.760442 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29333419-951b-45b8-b591-6e17bf85705d admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:50:41.760744 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29333419-951b-45b8-b591-6e17bf85705d admin admin] PATH_INFO: `/v3/users/f8022d5f904e4ae294b2cdf21aa11212` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:50:41.761903 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29333419-951b-45b8-b591-6e17bf85705d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8022d5f904e4ae294b2cdf21aa11212)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:50:41.763856 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29333419-951b-45b8-b591-6e17bf85705d admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:50:41.971534 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-29333419-951b-45b8-b591-6e17bf85705d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8022d5f904e4ae294b2cdf21aa11212'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:50:41.972257 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-29333419-951b-45b8-b591-6e17bf85705d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8022d5f904e4ae294b2cdf21aa11212'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:50:42.026993 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-29333419-951b-45b8-b591-6e17bf85705d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8022d5f904e4ae294b2cdf21aa11212'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:50:42.027645 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-29333419-951b-45b8-b591-6e17bf85705d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8022d5f904e4ae294b2cdf21aa11212'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:50:42.035314 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-29333419-951b-45b8-b591-6e17bf85705d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8022d5f904e4ae294b2cdf21aa11212'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:50:42.055307 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1377/2759] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:50:41 2022] DELETE /identity/v3/users/f8022d5f904e4ae294b2cdf21aa11212 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 07:50:42.348602 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bcf34d-e672-4077-91ea-f2401261f093 service placement] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:50:42.361334 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bcf34d-e672-4077-91ea-f2401261f093 service placement] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:50:42.419822 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc3b13c-cba8-4d8f-a849-d1a65cb9128d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:50:42.420455 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc3b13c-cba8-4d8f-a849-d1a65cb9128d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:50:42.420734 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc3b13c-cba8-4d8f-a849-d1a65cb9128d admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:50:42.421001 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc3b13c-cba8-4d8f-a849-d1a65cb9128d admin admin] PATH_INFO: `/v3/projects/a9d0f5e6d6cf423485157e1381a12713` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:50:42.422152 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc3b13c-cba8-4d8f-a849-d1a65cb9128d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9d0f5e6d6cf423485157e1381a12713)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:50:42.424048 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc3b13c-cba8-4d8f-a849-d1a65cb9128d admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:50:42.513776 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:50:42.523914 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-ccc3b13c-cba8-4d8f-a849-d1a65cb9128d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9d0f5e6d6cf423485157e1381a12713', 'request_id': 'req-ccc3b13c-cba8-4d8f-a849-d1a65cb9128d'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:50:42.530042 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-ccc3b13c-cba8-4d8f-a849-d1a65cb9128d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9d0f5e6d6cf423485157e1381a12713', 'request_id': 'req-ccc3b13c-cba8-4d8f-a849-d1a65cb9128d'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:50:42.609835 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-ccc3b13c-cba8-4d8f-a849-d1a65cb9128d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9d0f5e6d6cf423485157e1381a12713', 'request_id': 'req-ccc3b13c-cba8-4d8f-a849-d1a65cb9128d'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:50:42.620484 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-ccc3b13c-cba8-4d8f-a849-d1a65cb9128d admin admin] The token cache is being invalidate because project a9d0f5e6d6cf423485157e1381a12713 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:50:42.621342 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-ccc3b13c-cba8-4d8f-a849-d1a65cb9128d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:50:42.622156 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-ccc3b13c-cba8-4d8f-a849-d1a65cb9128d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:50:42.623650 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1383/2760] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:50:42 2022] DELETE /identity/v3/projects/a9d0f5e6d6cf423485157e1381a12713 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:50:42.628644 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03153df-4e30-4ff9-a15a-85c906ca9115 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:50:42.628932 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03153df-4e30-4ff9-a15a-85c906ca9115 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:50:42.629207 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03153df-4e30-4ff9-a15a-85c906ca9115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:50:42.650860 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-d03153df-4e30-4ff9-a15a-85c906ca9115 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:50:42.652059 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-d03153df-4e30-4ff9-a15a-85c906ca9115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:50:42.724549 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1378/2761] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:50:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:50:42.733639 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544b597d-2b32-41a1-8466-d7519090f37f None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:50:42.733935 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544b597d-2b32-41a1-8466-d7519090f37f None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:50:42.734218 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544b597d-2b32-41a1-8466-d7519090f37f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:50:42.754304 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-544b597d-2b32-41a1-8466-d7519090f37f None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:50:42.759080 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-544b597d-2b32-41a1-8466-d7519090f37f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:50:42.764181 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1384/2762] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:50:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:50:42.768410 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03153df-4e30-4ff9-a15a-85c906ca9115 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:50:42.778692 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03153df-4e30-4ff9-a15a-85c906ca9115 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:50:42.783257 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a3fbdf-d3dd-4292-b824-0bbf1dff3b2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:50:42.785364 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a3fbdf-d3dd-4292-b824-0bbf1dff3b2c admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:50:42.785364 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a3fbdf-d3dd-4292-b824-0bbf1dff3b2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:50:42.785364 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a3fbdf-d3dd-4292-b824-0bbf1dff3b2c admin admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:50:42.785364 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a3fbdf-d3dd-4292-b824-0bbf1dff3b2c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:50:42.785977 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a3fbdf-d3dd-4292-b824-0bbf1dff3b2c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:50:42.787259 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a3fbdf-d3dd-4292-b824-0bbf1dff3b2c admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:50:42.800141 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1379/2763] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 15 07:50:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:50:42.804467 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544b597d-2b32-41a1-8466-d7519090f37f None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:50:42.814023 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544b597d-2b32-41a1-8466-d7519090f37f None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:50:42.818896 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5878b4ba-40b8-4cc1-a5e8-15039b75dc77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:50:42.819508 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5878b4ba-40b8-4cc1-a5e8-15039b75dc77 admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:50:42.819788 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5878b4ba-40b8-4cc1-a5e8-15039b75dc77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:50:42.820063 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5878b4ba-40b8-4cc1-a5e8-15039b75dc77 admin admin] PATH_INFO: `/v3/projects` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:50:42.820701 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5878b4ba-40b8-4cc1-a5e8-15039b75dc77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:50:42.822638 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5878b4ba-40b8-4cc1-a5e8-15039b75dc77 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:50:42.914239 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1385/2764] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:50:42 2022] POST /identity/v3/projects => generated 367 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:50:42.918311 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a3fbdf-d3dd-4292-b824-0bbf1dff3b2c admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:50:42.929044 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a3fbdf-d3dd-4292-b824-0bbf1dff3b2c admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:50:42.932699 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:50:42.988301 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c25ec7-63d5-4849-ac2b-5502c5e6a56b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:50:42.989499 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c25ec7-63d5-4849-ac2b-5502c5e6a56b admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:50:42.989869 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c25ec7-63d5-4849-ac2b-5502c5e6a56b admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:50:42.990159 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c25ec7-63d5-4849-ac2b-5502c5e6a56b admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:50:42.990821 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c25ec7-63d5-4849-ac2b-5502c5e6a56b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:50:42.992721 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c25ec7-63d5-4849-ac2b-5502c5e6a56b admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:50:43.084930 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1380/2765] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:50:42 2022] POST /identity/v3/users => generated 284 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:50:43.089257 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5878b4ba-40b8-4cc1-a5e8-15039b75dc77 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:50:43.100154 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5878b4ba-40b8-4cc1-a5e8-15039b75dc77 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:50:43.104726 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db99ae6-ac18-4447-8f73-aec6bc60f09a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:50:43.105365 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db99ae6-ac18-4447-8f73-aec6bc60f09a admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:50:43.105676 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db99ae6-ac18-4447-8f73-aec6bc60f09a admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:50:43.105952 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db99ae6-ac18-4447-8f73-aec6bc60f09a admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:50:43.106580 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db99ae6-ac18-4447-8f73-aec6bc60f09a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:50:43.108163 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db99ae6-ac18-4447-8f73-aec6bc60f09a admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:50:43.118274 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1386/2766] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:50:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:50:43.122317 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c25ec7-63d5-4849-ac2b-5502c5e6a56b admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:50:43.132998 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c25ec7-63d5-4849-ac2b-5502c5e6a56b admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:50:43.137682 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d61195e-2824-4574-ba5a-413fa9543d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:50:43.138293 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d61195e-2824-4574-ba5a-413fa9543d1c admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:50:43.138579 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d61195e-2824-4574-ba5a-413fa9543d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:50:43.138857 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d61195e-2824-4574-ba5a-413fa9543d1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:50:43.139503 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d61195e-2824-4574-ba5a-413fa9543d1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:50:43.141090 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d61195e-2824-4574-ba5a-413fa9543d1c admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:50:43.152849 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1381/2767] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:50:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:50:43.157170 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db99ae6-ac18-4447-8f73-aec6bc60f09a admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:50:43.167722 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db99ae6-ac18-4447-8f73-aec6bc60f09a admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:50:43.172256 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe28f711-d03f-4465-870a-e7b558e5a5d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:50:43.172896 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe28f711-d03f-4465-870a-e7b558e5a5d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:50:43.173173 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe28f711-d03f-4465-870a-e7b558e5a5d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:50:43.173442 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe28f711-d03f-4465-870a-e7b558e5a5d3 admin admin] PATH_INFO: `/v3/projects/0fd6dbcf88bb4b5db5ca85927b6ab7c4/users/a13bb1edccf543379d6f57db833168fd/roles/053e12af84e943e699460a9d76c2072b` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:50:43.188204 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe28f711-d03f-4465-870a-e7b558e5a5d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fd6dbcf88bb4b5db5ca85927b6ab7c4, user_id=a13bb1edccf543379d6f57db833168fd, role_id=053e12af84e943e699460a9d76c2072b)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:50:43.190874 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe28f711-d03f-4465-870a-e7b558e5a5d3 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:50:43.290524 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1387/2768] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:50:43 2022] PUT /identity/v3/projects/0fd6dbcf88bb4b5db5ca85927b6ab7c4/users/a13bb1edccf543379d6f57db833168fd/roles/053e12af84e943e699460a9d76c2072b => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:50:43.300946 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304caa2b-1cb8-4e37-b678-9edac8614a61 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:50:43.302208 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304caa2b-1cb8-4e37-b678-9edac8614a61 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:50:43.302208 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304caa2b-1cb8-4e37-b678-9edac8614a61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:50:43.330591 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-304caa2b-1cb8-4e37-b678-9edac8614a61 None None] MFA Rules not processed for user `a13bb1edccf543379d6f57db833168fd`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:50:43.331791 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-304caa2b-1cb8-4e37-b678-9edac8614a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:50:43.394768 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1382/2769] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:50:43 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:50:43.399917 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe28f711-d03f-4465-870a-e7b558e5a5d3 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:50:43.462493 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe28f711-d03f-4465-870a-e7b558e5a5d3 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:50:43.467135 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd673e49-c88b-4b4e-87f2-efea0377d99b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:50:43.470448 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd673e49-c88b-4b4e-87f2-efea0377d99b admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:50:43.470448 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd673e49-c88b-4b4e-87f2-efea0377d99b admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:50:43.470448 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd673e49-c88b-4b4e-87f2-efea0377d99b admin admin] PATH_INFO: `/v3/projects` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:50:43.470448 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd673e49-c88b-4b4e-87f2-efea0377d99b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:50:43.470875 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd673e49-c88b-4b4e-87f2-efea0377d99b admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:50:43.542333 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1388/2770] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:50:43 2022] POST /identity/v3/projects => generated 369 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:50:43.546747 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304caa2b-1cb8-4e37-b678-9edac8614a61 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:50:43.557564 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304caa2b-1cb8-4e37-b678-9edac8614a61 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:50:43.624894 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f60e66-762c-4645-a615-9613290c98ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:50:43.625501 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f60e66-762c-4645-a615-9613290c98ff admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:50:43.625874 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f60e66-762c-4645-a615-9613290c98ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:50:43.626147 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f60e66-762c-4645-a615-9613290c98ff admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:50:43.626773 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f60e66-762c-4645-a615-9613290c98ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:50:43.628745 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f60e66-762c-4645-a615-9613290c98ff admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:50:43.711699 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1383/2771] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:50:43 2022] POST /identity/v3/users => generated 291 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:50:43.715648 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd673e49-c88b-4b4e-87f2-efea0377d99b admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:50:43.727137 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd673e49-c88b-4b4e-87f2-efea0377d99b admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:50:43.731946 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02511f2-0f8a-4aeb-a587-ea9b98dc862c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:50:43.732270 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02511f2-0f8a-4aeb-a587-ea9b98dc862c admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:50:43.732572 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02511f2-0f8a-4aeb-a587-ea9b98dc862c admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:50:43.732842 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02511f2-0f8a-4aeb-a587-ea9b98dc862c admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:50:43.733605 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02511f2-0f8a-4aeb-a587-ea9b98dc862c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:50:43.735218 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02511f2-0f8a-4aeb-a587-ea9b98dc862c admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:50:43.746555 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1389/2772] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:50:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:50:43.750613 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f60e66-762c-4645-a615-9613290c98ff admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:50:43.757817 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:50:43.761508 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f60e66-762c-4645-a615-9613290c98ff admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:50:43.766185 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2719a6-6aa0-418d-bc62-b640687e95bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:50:43.766869 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2719a6-6aa0-418d-bc62-b640687e95bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:50:43.767146 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2719a6-6aa0-418d-bc62-b640687e95bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:50:43.767420 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2719a6-6aa0-418d-bc62-b640687e95bb admin admin] PATH_INFO: `/v3/projects/8abb21b34cf94c33adaa1e3eaa05bb8d/users/e0c7b5ba63404d0494180af99ea36805/roles/6bbb643e0d8d4bc0a9b27c26edf59216` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:50:43.782343 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2719a6-6aa0-418d-bc62-b640687e95bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=8abb21b34cf94c33adaa1e3eaa05bb8d, user_id=e0c7b5ba63404d0494180af99ea36805, role_id=6bbb643e0d8d4bc0a9b27c26edf59216)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:50:43.785006 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2719a6-6aa0-418d-bc62-b640687e95bb admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:50:43.877903 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1384/2773] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:50:43 2022] PUT /identity/v3/projects/8abb21b34cf94c33adaa1e3eaa05bb8d/users/e0c7b5ba63404d0494180af99ea36805/roles/6bbb643e0d8d4bc0a9b27c26edf59216 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:50:43.882093 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02511f2-0f8a-4aeb-a587-ea9b98dc862c admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:50:43.897880 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02511f2-0f8a-4aeb-a587-ea9b98dc862c admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:50:43.921438 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:50:43.964488 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74458531-9d24-42be-9efe-db84e6a72ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:50:43.964950 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74458531-9d24-42be-9efe-db84e6a72ace admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:50:43.965233 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74458531-9d24-42be-9efe-db84e6a72ace admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:50:43.965510 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74458531-9d24-42be-9efe-db84e6a72ace admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:50:43.966191 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74458531-9d24-42be-9efe-db84e6a72ace admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:50:43.967780 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74458531-9d24-42be-9efe-db84e6a72ace admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:50:43.978208 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1390/2774] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:50:43 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:50:43.982399 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2719a6-6aa0-418d-bc62-b640687e95bb admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:50:43.993316 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2719a6-6aa0-418d-bc62-b640687e95bb admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:50:43.997954 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94f3005-e229-4339-aec8-2c8eafa6cbb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:50:43.998600 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94f3005-e229-4339-aec8-2c8eafa6cbb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:50:43.998879 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94f3005-e229-4339-aec8-2c8eafa6cbb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:50:43.999153 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94f3005-e229-4339-aec8-2c8eafa6cbb9 admin admin] PATH_INFO: `/v3/projects/8abb21b34cf94c33adaa1e3eaa05bb8d/users/e0c7b5ba63404d0494180af99ea36805/roles/6bbb643e0d8d4bc0a9b27c26edf59216` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:50:44.001088 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94f3005-e229-4339-aec8-2c8eafa6cbb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8abb21b34cf94c33adaa1e3eaa05bb8d, user_id=e0c7b5ba63404d0494180af99ea36805, role_id=6bbb643e0d8d4bc0a9b27c26edf59216)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:50:44.003730 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94f3005-e229-4339-aec8-2c8eafa6cbb9 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:50:44.014956 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1385/2775] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:50:43 2022] PUT /identity/v3/projects/8abb21b34cf94c33adaa1e3eaa05bb8d/users/e0c7b5ba63404d0494180af99ea36805/roles/6bbb643e0d8d4bc0a9b27c26edf59216 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:50:44.025259 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8d12a2-4b41-4430-a6d1-67f477a026d8 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:50:44.025550 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8d12a2-4b41-4430-a6d1-67f477a026d8 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:50:44.025862 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8d12a2-4b41-4430-a6d1-67f477a026d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:50:44.054697 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-4b8d12a2-4b41-4430-a6d1-67f477a026d8 None None] MFA Rules not processed for user `e0c7b5ba63404d0494180af99ea36805`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:50:44.055900 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-4b8d12a2-4b41-4430-a6d1-67f477a026d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:50:44.123962 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1391/2776] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:50:44 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 07:50:44.136128 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94f3005-e229-4339-aec8-2c8eafa6cbb9 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:50:44.137827 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-a94f3005-e229-4339-aec8-2c8eafa6cbb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:50:44.195694 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94f3005-e229-4339-aec8-2c8eafa6cbb9 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:50:44.199722 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7354031-1a1e-43b7-b008-22a3043cd7c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecf6642b7b84d3c9756bb41ab3349d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:50:44.200340 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7354031-1a1e-43b7-b008-22a3043cd7c3 service cinder] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:50:44.200626 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7354031-1a1e-43b7-b008-22a3043cd7c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:50:44.200901 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7354031-1a1e-43b7-b008-22a3043cd7c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:50:44.215282 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7354031-1a1e-43b7-b008-22a3043cd7c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:50:44.216858 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7354031-1a1e-43b7-b008-22a3043cd7c3 service cinder] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:50:44.227072 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1386/2777] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:50:44 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:50:46.748814 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8d12a2-4b41-4430-a6d1-67f477a026d8 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:50:46.810773 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8d12a2-4b41-4430-a6d1-67f477a026d8 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:50:46.815661 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf47e0c-0768-43f8-b56d-7702b612a603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:50:46.816303 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf47e0c-0768-43f8-b56d-7702b612a603 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:50:46.816594 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf47e0c-0768-43f8-b56d-7702b612a603 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:50:46.816873 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf47e0c-0768-43f8-b56d-7702b612a603 admin admin] PATH_INFO: `/v3/users/a13bb1edccf543379d6f57db833168fd` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:50:46.818067 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf47e0c-0768-43f8-b56d-7702b612a603 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a13bb1edccf543379d6f57db833168fd)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:50:46.820881 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf47e0c-0768-43f8-b56d-7702b612a603 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:50:46.993369 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-aaf47e0c-0768-43f8-b56d-7702b612a603 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a13bb1edccf543379d6f57db833168fd'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:50:46.994984 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:50:47.054760 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-aaf47e0c-0768-43f8-b56d-7702b612a603 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a13bb1edccf543379d6f57db833168fd'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:50:47.055375 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-aaf47e0c-0768-43f8-b56d-7702b612a603 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a13bb1edccf543379d6f57db833168fd'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:50:47.055978 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-aaf47e0c-0768-43f8-b56d-7702b612a603 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a13bb1edccf543379d6f57db833168fd'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:50:47.063223 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-aaf47e0c-0768-43f8-b56d-7702b612a603 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a13bb1edccf543379d6f57db833168fd'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:50:47.080043 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1392/2778] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:50:46 2022] DELETE /identity/v3/users/a13bb1edccf543379d6f57db833168fd => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 07:50:47.085001 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aec0ca-b1a0-44d6-86d7-6486c9ca9078 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:50:47.085484 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aec0ca-b1a0-44d6-86d7-6486c9ca9078 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:50:47.085629 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aec0ca-b1a0-44d6-86d7-6486c9ca9078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:50:47.104643 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-02aec0ca-b1a0-44d6-86d7-6486c9ca9078 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:50:47.105856 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-02aec0ca-b1a0-44d6-86d7-6486c9ca9078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:50:47.164639 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1387/2779] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:50:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:50:47.174319 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf47e0c-0768-43f8-b56d-7702b612a603 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:50:47.175725 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-aaf47e0c-0768-43f8-b56d-7702b612a603 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:50:47.233627 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf47e0c-0768-43f8-b56d-7702b612a603 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:50:47.237296 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d76fbb7-d609-4755-9d11-0105407e8ee5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:50:47.237932 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d76fbb7-d609-4755-9d11-0105407e8ee5 service neutron] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:50:47.238222 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d76fbb7-d609-4755-9d11-0105407e8ee5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:50:47.238500 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d76fbb7-d609-4755-9d11-0105407e8ee5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:50:47.248389 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d76fbb7-d609-4755-9d11-0105407e8ee5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:50:47.249792 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d76fbb7-d609-4755-9d11-0105407e8ee5 service neutron] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:50:47.259226 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1393/2780] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:50:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 07:50:47.669796 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aec0ca-b1a0-44d6-86d7-6486c9ca9078 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:50:47.669796 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-02aec0ca-b1a0-44d6-86d7-6486c9ca9078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:50:47.676981 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aec0ca-b1a0-44d6-86d7-6486c9ca9078 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:50:47.681920 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6720e517-b3c3-4716-8d3c-cdce549980f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:50:47.682598 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6720e517-b3c3-4716-8d3c-cdce549980f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:50:47.682882 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6720e517-b3c3-4716-8d3c-cdce549980f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:50:47.683155 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6720e517-b3c3-4716-8d3c-cdce549980f9 admin admin] PATH_INFO: `/v3/projects/0fd6dbcf88bb4b5db5ca85927b6ab7c4` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:50:47.684293 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6720e517-b3c3-4716-8d3c-cdce549980f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fd6dbcf88bb4b5db5ca85927b6ab7c4)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:50:47.686241 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6720e517-b3c3-4716-8d3c-cdce549980f9 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:50:47.822094 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-6720e517-b3c3-4716-8d3c-cdce549980f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fd6dbcf88bb4b5db5ca85927b6ab7c4', 'request_id': 'req-6720e517-b3c3-4716-8d3c-cdce549980f9'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:50:47.828384 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-6720e517-b3c3-4716-8d3c-cdce549980f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fd6dbcf88bb4b5db5ca85927b6ab7c4', 'request_id': 'req-6720e517-b3c3-4716-8d3c-cdce549980f9'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:50:47.894964 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-6720e517-b3c3-4716-8d3c-cdce549980f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fd6dbcf88bb4b5db5ca85927b6ab7c4', 'request_id': 'req-6720e517-b3c3-4716-8d3c-cdce549980f9'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:50:47.899567 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:50:47.906559 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-6720e517-b3c3-4716-8d3c-cdce549980f9 admin admin] The token cache is being invalidate because project 0fd6dbcf88bb4b5db5ca85927b6ab7c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:50:47.906855 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-6720e517-b3c3-4716-8d3c-cdce549980f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:50:47.907481 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-6720e517-b3c3-4716-8d3c-cdce549980f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:50:47.908970 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1388/2781] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:50:47 2022] DELETE /identity/v3/projects/0fd6dbcf88bb4b5db5ca85927b6ab7c4 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 07:50:47.913158 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d76fbb7-d609-4755-9d11-0105407e8ee5 service neutron] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:50:47.922887 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d76fbb7-d609-4755-9d11-0105407e8ee5 service neutron] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:50:47.980192 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0da553a-f9d7-4302-a4d3-294855168bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:50:47.980828 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da553a-f9d7-4302-a4d3-294855168bb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:50:47.981110 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da553a-f9d7-4302-a4d3-294855168bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:50:47.981389 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da553a-f9d7-4302-a4d3-294855168bb4 admin admin] PATH_INFO: `/v3/users/e0c7b5ba63404d0494180af99ea36805` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:50:47.982545 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0da553a-f9d7-4302-a4d3-294855168bb4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0c7b5ba63404d0494180af99ea36805)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:50:47.984465 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0da553a-f9d7-4302-a4d3-294855168bb4 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:50:48.154476 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-a0da553a-f9d7-4302-a4d3-294855168bb4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0c7b5ba63404d0494180af99ea36805'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:50:48.242595 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-a0da553a-f9d7-4302-a4d3-294855168bb4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0c7b5ba63404d0494180af99ea36805'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:50:48.243257 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-a0da553a-f9d7-4302-a4d3-294855168bb4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0c7b5ba63404d0494180af99ea36805'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:50:48.243876 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-a0da553a-f9d7-4302-a4d3-294855168bb4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0c7b5ba63404d0494180af99ea36805'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:50:48.251384 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-a0da553a-f9d7-4302-a4d3-294855168bb4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0c7b5ba63404d0494180af99ea36805'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:50:48.257587 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:INSERT=2,keystone:DELETE=12,keystone:SELECT=86 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:50:48.269871 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1394/2782] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:50:47 2022] DELETE /identity/v3/users/e0c7b5ba63404d0494180af99ea36805 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:50:48.633721 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6720e517-b3c3-4716-8d3c-cdce549980f9 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:50:48.635195 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-6720e517-b3c3-4716-8d3c-cdce549980f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:50:48.648896 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6720e517-b3c3-4716-8d3c-cdce549980f9 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:50:48.706050 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe62ad42-a807-44bd-a862-8f628d06ab49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:50:48.706603 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe62ad42-a807-44bd-a862-8f628d06ab49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:50:48.706898 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe62ad42-a807-44bd-a862-8f628d06ab49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:50:48.707151 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe62ad42-a807-44bd-a862-8f628d06ab49 admin admin] PATH_INFO: `/v3/projects/8abb21b34cf94c33adaa1e3eaa05bb8d` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:50:48.708276 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe62ad42-a807-44bd-a862-8f628d06ab49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8abb21b34cf94c33adaa1e3eaa05bb8d)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:50:48.710468 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe62ad42-a807-44bd-a862-8f628d06ab49 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:50:48.826802 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-fe62ad42-a807-44bd-a862-8f628d06ab49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8abb21b34cf94c33adaa1e3eaa05bb8d', 'request_id': 'req-fe62ad42-a807-44bd-a862-8f628d06ab49'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:50:48.832904 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-fe62ad42-a807-44bd-a862-8f628d06ab49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8abb21b34cf94c33adaa1e3eaa05bb8d', 'request_id': 'req-fe62ad42-a807-44bd-a862-8f628d06ab49'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:50:48.894861 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-fe62ad42-a807-44bd-a862-8f628d06ab49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8abb21b34cf94c33adaa1e3eaa05bb8d', 'request_id': 'req-fe62ad42-a807-44bd-a862-8f628d06ab49'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:50:48.906007 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-fe62ad42-a807-44bd-a862-8f628d06ab49 admin admin] The token cache is being invalidate because project 8abb21b34cf94c33adaa1e3eaa05bb8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:50:48.906301 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-fe62ad42-a807-44bd-a862-8f628d06ab49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:50:48.906922 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-fe62ad42-a807-44bd-a862-8f628d06ab49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:50:48.908407 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1389/2783] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:50:48 2022] DELETE /identity/v3/projects/8abb21b34cf94c33adaa1e3eaa05bb8d => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:50:48.913939 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4952490f-c881-4dc7-9577-fc925c6b7a8f None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:50:48.914438 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4952490f-c881-4dc7-9577-fc925c6b7a8f None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:50:48.914539 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4952490f-c881-4dc7-9577-fc925c6b7a8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:50:48.933809 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-4952490f-c881-4dc7-9577-fc925c6b7a8f None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:50:48.934994 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-4952490f-c881-4dc7-9577-fc925c6b7a8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:50:48.994254 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1395/2784] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:50:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:50:49.002062 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8975c529-20d5-4ad8-b3fd-21629a2d3abe None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:50:49.002345 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8975c529-20d5-4ad8-b3fd-21629a2d3abe None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:50:49.002617 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8975c529-20d5-4ad8-b3fd-21629a2d3abe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:50:49.022868 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-8975c529-20d5-4ad8-b3fd-21629a2d3abe None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:50:49.024056 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-8975c529-20d5-4ad8-b3fd-21629a2d3abe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:50:49.031729 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1390/2785] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:50:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:50:49.035965 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4952490f-c881-4dc7-9577-fc925c6b7a8f None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:50:49.046329 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4952490f-c881-4dc7-9577-fc925c6b7a8f None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:50:49.050789 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6da89a-6373-4c50-8a3d-b029410e4f49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:50:49.051665 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6da89a-6373-4c50-8a3d-b029410e4f49 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:50:49.051945 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6da89a-6373-4c50-8a3d-b029410e4f49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:50:49.052215 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6da89a-6373-4c50-8a3d-b029410e4f49 admin admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:50:49.053131 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6da89a-6373-4c50-8a3d-b029410e4f49 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:50:49.053384 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6da89a-6373-4c50-8a3d-b029410e4f49 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:50:49.055011 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6da89a-6373-4c50-8a3d-b029410e4f49 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:50:49.066404 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1396/2786] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 15 07:50:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:50:49.070579 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8975c529-20d5-4ad8-b3fd-21629a2d3abe None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:50:49.079549 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8975c529-20d5-4ad8-b3fd-21629a2d3abe None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:50:49.085283 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9646b0-ea9b-4530-99f0-bcc5815119d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:50:49.085917 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9646b0-ea9b-4530-99f0-bcc5815119d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:50:49.086202 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9646b0-ea9b-4530-99f0-bcc5815119d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:50:49.086480 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9646b0-ea9b-4530-99f0-bcc5815119d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:50:49.087109 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9646b0-ea9b-4530-99f0-bcc5815119d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:50:49.088984 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9646b0-ea9b-4530-99f0-bcc5815119d8 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:50:49.181989 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1391/2787] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:50:49 2022] POST /identity/v3/projects => generated 369 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 07:50:49.186158 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6da89a-6373-4c50-8a3d-b029410e4f49 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:50:49.198114 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6da89a-6373-4c50-8a3d-b029410e4f49 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:50:49.256317 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4cac4f-4d22-4c84-838a-047e4af5a097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:50:49.256844 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4cac4f-4d22-4c84-838a-047e4af5a097 admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:50:49.257117 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4cac4f-4d22-4c84-838a-047e4af5a097 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:50:49.257386 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4cac4f-4d22-4c84-838a-047e4af5a097 admin admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:50:49.258062 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4cac4f-4d22-4c84-838a-047e4af5a097 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:50:49.259955 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4cac4f-4d22-4c84-838a-047e4af5a097 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:50:49.317397 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1397/2788] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:50:49 2022] POST /identity/v3/users => generated 285 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:50:49.322257 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9646b0-ea9b-4530-99f0-bcc5815119d8 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:50:49.332437 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9646b0-ea9b-4530-99f0-bcc5815119d8 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:50:49.337900 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ecaab1-7d64-443b-b428-f415fb70de37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:50:49.338487 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ecaab1-7d64-443b-b428-f415fb70de37 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:50:49.338763 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ecaab1-7d64-443b-b428-f415fb70de37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:50:49.339031 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ecaab1-7d64-443b-b428-f415fb70de37 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:50:49.339662 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ecaab1-7d64-443b-b428-f415fb70de37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:50:49.341223 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ecaab1-7d64-443b-b428-f415fb70de37 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:50:49.351199 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1392/2789] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:50:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:50:49.355121 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4cac4f-4d22-4c84-838a-047e4af5a097 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:50:49.365022 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4cac4f-4d22-4c84-838a-047e4af5a097 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:50:49.369873 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5317272f-9e3f-45e3-a447-577b5fc01590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:50:49.370472 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5317272f-9e3f-45e3-a447-577b5fc01590 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:50:49.370755 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5317272f-9e3f-45e3-a447-577b5fc01590 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:50:49.371032 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5317272f-9e3f-45e3-a447-577b5fc01590 admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:50:49.371664 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5317272f-9e3f-45e3-a447-577b5fc01590 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:50:49.373248 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5317272f-9e3f-45e3-a447-577b5fc01590 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:50:49.384365 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1398/2790] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:50:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:50:49.388340 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ecaab1-7d64-443b-b428-f415fb70de37 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:50:49.398632 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ecaab1-7d64-443b-b428-f415fb70de37 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:50:49.403153 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfee232d-46bb-4ab8-9c17-9bc506f8f100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:50:49.403791 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfee232d-46bb-4ab8-9c17-9bc506f8f100 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:50:49.404068 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfee232d-46bb-4ab8-9c17-9bc506f8f100 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:50:49.404340 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfee232d-46bb-4ab8-9c17-9bc506f8f100 admin admin] PATH_INFO: `/v3/projects/ecfff1b0ba9d499695914391a70c4e00/users/6c92482dea164a28a9e8b5f1cb70403e/roles/053e12af84e943e699460a9d76c2072b` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:50:49.419346 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfee232d-46bb-4ab8-9c17-9bc506f8f100 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecfff1b0ba9d499695914391a70c4e00, user_id=6c92482dea164a28a9e8b5f1cb70403e, role_id=053e12af84e943e699460a9d76c2072b)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:50:49.422013 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfee232d-46bb-4ab8-9c17-9bc506f8f100 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:50:49.483464 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1393/2791] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:50:49 2022] PUT /identity/v3/projects/ecfff1b0ba9d499695914391a70c4e00/users/6c92482dea164a28a9e8b5f1cb70403e/roles/053e12af84e943e699460a9d76c2072b => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 07:50:49.494208 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3aea19-c117-416c-bb8e-9fe26308f0a7 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:50:49.494512 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3aea19-c117-416c-bb8e-9fe26308f0a7 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:50:49.494787 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3aea19-c117-416c-bb8e-9fe26308f0a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:50:49.523274 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-4a3aea19-c117-416c-bb8e-9fe26308f0a7 None None] MFA Rules not processed for user `6c92482dea164a28a9e8b5f1cb70403e`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:50:49.524395 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-4a3aea19-c117-416c-bb8e-9fe26308f0a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:50:49.564243 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:50:49.587942 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1399/2792] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:50:49 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:50:49.592572 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfee232d-46bb-4ab8-9c17-9bc506f8f100 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:50:49.613766 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:50:49.656425 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfee232d-46bb-4ab8-9c17-9bc506f8f100 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:50:49.661415 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0a2908-dfb9-40ad-85a7-7d31b99cb231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:50:49.662105 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0a2908-dfb9-40ad-85a7-7d31b99cb231 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:50:49.662482 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0a2908-dfb9-40ad-85a7-7d31b99cb231 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:50:49.662764 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0a2908-dfb9-40ad-85a7-7d31b99cb231 admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:50:49.663402 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0a2908-dfb9-40ad-85a7-7d31b99cb231 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:50:49.665297 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0a2908-dfb9-40ad-85a7-7d31b99cb231 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:50:49.748271 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1394/2793] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:50:49 2022] POST /identity/v3/projects => generated 369 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:50:49.752467 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3aea19-c117-416c-bb8e-9fe26308f0a7 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:50:49.764062 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3aea19-c117-416c-bb8e-9fe26308f0a7 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:50:49.823457 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800d8c96-a71a-40a3-8f13-614ed1e8cd7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:50:49.824071 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800d8c96-a71a-40a3-8f13-614ed1e8cd7d admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:50:49.824344 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800d8c96-a71a-40a3-8f13-614ed1e8cd7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:50:49.824613 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800d8c96-a71a-40a3-8f13-614ed1e8cd7d admin admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:50:49.825244 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800d8c96-a71a-40a3-8f13-614ed1e8cd7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:50:49.827181 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800d8c96-a71a-40a3-8f13-614ed1e8cd7d admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:50:49.901365 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1400/2794] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:50:49 2022] POST /identity/v3/users => generated 291 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:50:49.905379 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0a2908-dfb9-40ad-85a7-7d31b99cb231 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:50:49.915924 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0a2908-dfb9-40ad-85a7-7d31b99cb231 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:50:49.920641 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7415173f-ce31-4167-9074-6f1998534a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:50:49.921235 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7415173f-ce31-4167-9074-6f1998534a4b admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:50:49.921510 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7415173f-ce31-4167-9074-6f1998534a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:50:49.921811 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7415173f-ce31-4167-9074-6f1998534a4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:50:49.922445 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7415173f-ce31-4167-9074-6f1998534a4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:50:49.924040 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7415173f-ce31-4167-9074-6f1998534a4b admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:50:49.935239 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1395/2795] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:50:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:50:49.939415 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800d8c96-a71a-40a3-8f13-614ed1e8cd7d admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:50:49.950631 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800d8c96-a71a-40a3-8f13-614ed1e8cd7d admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:50:49.955302 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673a7a29-03d3-40f3-94e1-2acd005b6d79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:50:49.955955 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673a7a29-03d3-40f3-94e1-2acd005b6d79 admin admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:50:49.956260 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673a7a29-03d3-40f3-94e1-2acd005b6d79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:50:49.956530 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673a7a29-03d3-40f3-94e1-2acd005b6d79 admin admin] PATH_INFO: `/v3/projects/99843c7e351145f8bfa979be14a15d70/users/57b9a8aa2abd4df18b339160af1acdc0/roles/6bbb643e0d8d4bc0a9b27c26edf59216` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:50:49.971682 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673a7a29-03d3-40f3-94e1-2acd005b6d79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99843c7e351145f8bfa979be14a15d70, user_id=57b9a8aa2abd4df18b339160af1acdc0, role_id=6bbb643e0d8d4bc0a9b27c26edf59216)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:50:49.974433 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673a7a29-03d3-40f3-94e1-2acd005b6d79 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:50:50.043595 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1401/2796] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:50:49 2022] PUT /identity/v3/projects/99843c7e351145f8bfa979be14a15d70/users/57b9a8aa2abd4df18b339160af1acdc0/roles/6bbb643e0d8d4bc0a9b27c26edf59216 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:50:50.047476 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7415173f-ce31-4167-9074-6f1998534a4b admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:50:50.058785 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7415173f-ce31-4167-9074-6f1998534a4b admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:50:50.117782 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65f533a-1a7e-4ce9-82c8-57efbb7af081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:50:50.118850 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65f533a-1a7e-4ce9-82c8-57efbb7af081 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:50:50.118850 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65f533a-1a7e-4ce9-82c8-57efbb7af081 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:50:50.118850 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65f533a-1a7e-4ce9-82c8-57efbb7af081 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:50:50.119452 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65f533a-1a7e-4ce9-82c8-57efbb7af081 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:50:50.121087 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65f533a-1a7e-4ce9-82c8-57efbb7af081 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:50:50.131546 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1396/2797] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:50:50 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 07:50:50.135612 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673a7a29-03d3-40f3-94e1-2acd005b6d79 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:50:50.145830 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673a7a29-03d3-40f3-94e1-2acd005b6d79 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:50:50.150995 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a983c1-3d28-4365-a03d-7f13d1e38b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:50:50.151738 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a983c1-3d28-4365-a03d-7f13d1e38b46 admin admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:50:50.152020 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a983c1-3d28-4365-a03d-7f13d1e38b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:50:50.152956 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a983c1-3d28-4365-a03d-7f13d1e38b46 admin admin] PATH_INFO: `/v3/projects/99843c7e351145f8bfa979be14a15d70/users/57b9a8aa2abd4df18b339160af1acdc0/roles/6bbb643e0d8d4bc0a9b27c26edf59216` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:50:50.155245 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a983c1-3d28-4365-a03d-7f13d1e38b46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99843c7e351145f8bfa979be14a15d70, user_id=57b9a8aa2abd4df18b339160af1acdc0, role_id=6bbb643e0d8d4bc0a9b27c26edf59216)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:50:50.157862 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a983c1-3d28-4365-a03d-7f13d1e38b46 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:50:50.169649 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1402/2798] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:50:50 2022] PUT /identity/v3/projects/99843c7e351145f8bfa979be14a15d70/users/57b9a8aa2abd4df18b339160af1acdc0/roles/6bbb643e0d8d4bc0a9b27c26edf59216 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:50:50.181569 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd640e59-323f-4c94-8859-9921acf12653 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:50:50.181894 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd640e59-323f-4c94-8859-9921acf12653 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:50:50.182175 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd640e59-323f-4c94-8859-9921acf12653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:50:50.210072 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-bd640e59-323f-4c94-8859-9921acf12653 None None] MFA Rules not processed for user `57b9a8aa2abd4df18b339160af1acdc0`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:50:50.211287 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-bd640e59-323f-4c94-8859-9921acf12653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:50:50.280424 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1397/2799] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:50:50 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:50:50.292106 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a983c1-3d28-4365-a03d-7f13d1e38b46 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:50:50.293791 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-55a983c1-3d28-4365-a03d-7f13d1e38b46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:50:50.351434 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a983c1-3d28-4365-a03d-7f13d1e38b46 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:50:50.355075 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6074dcc7-3428-415e-99fe-01a85c7df3e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecf6642b7b84d3c9756bb41ab3349d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:50:50.355678 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6074dcc7-3428-415e-99fe-01a85c7df3e0 service cinder] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:50:50.355959 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6074dcc7-3428-415e-99fe-01a85c7df3e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:50:50.356232 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6074dcc7-3428-415e-99fe-01a85c7df3e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:50:50.365782 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6074dcc7-3428-415e-99fe-01a85c7df3e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:50:50.367425 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6074dcc7-3428-415e-99fe-01a85c7df3e0 service cinder] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:50:50.378054 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1403/2800] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:50:50 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:50:50.493053 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd640e59-323f-4c94-8859-9921acf12653 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:50:50.494507 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-bd640e59-323f-4c94-8859-9921acf12653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:50:50.506316 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd640e59-323f-4c94-8859-9921acf12653 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:50:50.509852 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9350503d-5b48-44fe-9b4d-a69f2d4dee3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecf6642b7b84d3c9756bb41ab3349d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:50:50.510451 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9350503d-5b48-44fe-9b4d-a69f2d4dee3f service cinder] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:50:50.511651 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9350503d-5b48-44fe-9b4d-a69f2d4dee3f service cinder] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:50:50.511651 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9350503d-5b48-44fe-9b4d-a69f2d4dee3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:50:50.523819 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:50:50.568141 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9350503d-5b48-44fe-9b4d-a69f2d4dee3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:50:50.569910 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9350503d-5b48-44fe-9b4d-a69f2d4dee3f service cinder] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:50:50.578598 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1398/2801] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:50:50 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:50:55.366059 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6074dcc7-3428-415e-99fe-01a85c7df3e0 service cinder] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:50:55.407140 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:50:55.428135 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6074dcc7-3428-415e-99fe-01a85c7df3e0 service cinder] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:50:55.432793 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdb4917-0f17-4919-8d76-90e0d39807b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:50:55.433418 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdb4917-0f17-4919-8d76-90e0d39807b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:50:55.433799 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdb4917-0f17-4919-8d76-90e0d39807b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:50:55.434113 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdb4917-0f17-4919-8d76-90e0d39807b2 admin admin] PATH_INFO: `/v3/users/6c92482dea164a28a9e8b5f1cb70403e` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:50:55.439562 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdb4917-0f17-4919-8d76-90e0d39807b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c92482dea164a28a9e8b5f1cb70403e)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:50:55.439562 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdb4917-0f17-4919-8d76-90e0d39807b2 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:50:55.609962 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-2fdb4917-0f17-4919-8d76-90e0d39807b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c92482dea164a28a9e8b5f1cb70403e'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:50:55.687311 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-2fdb4917-0f17-4919-8d76-90e0d39807b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c92482dea164a28a9e8b5f1cb70403e'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:50:55.687937 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-2fdb4917-0f17-4919-8d76-90e0d39807b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c92482dea164a28a9e8b5f1cb70403e'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:50:55.688552 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-2fdb4917-0f17-4919-8d76-90e0d39807b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c92482dea164a28a9e8b5f1cb70403e'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:50:55.696866 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-2fdb4917-0f17-4919-8d76-90e0d39807b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c92482dea164a28a9e8b5f1cb70403e'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:50:55.714395 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1404/2802] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:50:55 2022] DELETE /identity/v3/users/6c92482dea164a28a9e8b5f1cb70403e => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:50:55.718882 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5879e48-889a-4046-b83d-e7ebe32d9329 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:50:55.719384 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5879e48-889a-4046-b83d-e7ebe32d9329 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:50:55.719491 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5879e48-889a-4046-b83d-e7ebe32d9329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:50:55.738407 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-f5879e48-889a-4046-b83d-e7ebe32d9329 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:50:55.749565 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-f5879e48-889a-4046-b83d-e7ebe32d9329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:50:55.797240 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1399/2803] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:50:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:50:55.807537 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdb4917-0f17-4919-8d76-90e0d39807b2 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:50:55.808966 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-2fdb4917-0f17-4919-8d76-90e0d39807b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:50:55.866707 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdb4917-0f17-4919-8d76-90e0d39807b2 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:50:55.870207 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5e75cc-2846-4b61-ac24-a6e21a5c8cdd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:50:55.870801 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5e75cc-2846-4b61-ac24-a6e21a5c8cdd service neutron] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:50:55.871082 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5e75cc-2846-4b61-ac24-a6e21a5c8cdd service neutron] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:50:55.871354 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5e75cc-2846-4b61-ac24-a6e21a5c8cdd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:50:55.880023 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5e75cc-2846-4b61-ac24-a6e21a5c8cdd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:50:55.882347 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5e75cc-2846-4b61-ac24-a6e21a5c8cdd service neutron] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:50:55.891468 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1405/2804] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:50:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:50:56.330539 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5879e48-889a-4046-b83d-e7ebe32d9329 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:50:56.332143 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-f5879e48-889a-4046-b83d-e7ebe32d9329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:50:56.345697 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5879e48-889a-4046-b83d-e7ebe32d9329 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:50:56.350611 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1420f55f-1d74-4b92-9b4d-f06389dbe335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:50:56.351147 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1420f55f-1d74-4b92-9b4d-f06389dbe335 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:50:56.351428 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1420f55f-1d74-4b92-9b4d-f06389dbe335 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:50:56.351699 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1420f55f-1d74-4b92-9b4d-f06389dbe335 admin admin] PATH_INFO: `/v3/projects/ecfff1b0ba9d499695914391a70c4e00` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:50:56.352851 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1420f55f-1d74-4b92-9b4d-f06389dbe335 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ecfff1b0ba9d499695914391a70c4e00)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:50:56.354781 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1420f55f-1d74-4b92-9b4d-f06389dbe335 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:50:56.448356 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-1420f55f-1d74-4b92-9b4d-f06389dbe335 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ecfff1b0ba9d499695914391a70c4e00', 'request_id': 'req-1420f55f-1d74-4b92-9b4d-f06389dbe335'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:50:56.454375 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-1420f55f-1d74-4b92-9b4d-f06389dbe335 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ecfff1b0ba9d499695914391a70c4e00', 'request_id': 'req-1420f55f-1d74-4b92-9b4d-f06389dbe335'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:50:56.511482 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-1420f55f-1d74-4b92-9b4d-f06389dbe335 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ecfff1b0ba9d499695914391a70c4e00', 'request_id': 'req-1420f55f-1d74-4b92-9b4d-f06389dbe335'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:50:56.522643 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-1420f55f-1d74-4b92-9b4d-f06389dbe335 admin admin] The token cache is being invalidate because project ecfff1b0ba9d499695914391a70c4e00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:50:56.522936 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-1420f55f-1d74-4b92-9b4d-f06389dbe335 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:50:56.523570 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-1420f55f-1d74-4b92-9b4d-f06389dbe335 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:50:56.525382 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1400/2805] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:50:56 2022] DELETE /identity/v3/projects/ecfff1b0ba9d499695914391a70c4e00 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:50:56.529440 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5e75cc-2846-4b61-ac24-a6e21a5c8cdd service neutron] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:50:56.539244 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5e75cc-2846-4b61-ac24-a6e21a5c8cdd service neutron] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:50:56.595501 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e49d30d-8a42-47b6-bfa7-bc864564df4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:50:56.596120 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e49d30d-8a42-47b6-bfa7-bc864564df4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:50:56.596397 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e49d30d-8a42-47b6-bfa7-bc864564df4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:50:56.596668 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e49d30d-8a42-47b6-bfa7-bc864564df4f admin admin] PATH_INFO: `/v3/users/57b9a8aa2abd4df18b339160af1acdc0` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:50:56.597866 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e49d30d-8a42-47b6-bfa7-bc864564df4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=57b9a8aa2abd4df18b339160af1acdc0)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:50:56.600025 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e49d30d-8a42-47b6-bfa7-bc864564df4f admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:50:56.601527 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:50:56.822110 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-1e49d30d-8a42-47b6-bfa7-bc864564df4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57b9a8aa2abd4df18b339160af1acdc0'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:50:56.900134 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-1e49d30d-8a42-47b6-bfa7-bc864564df4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57b9a8aa2abd4df18b339160af1acdc0'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:50:56.900758 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-1e49d30d-8a42-47b6-bfa7-bc864564df4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57b9a8aa2abd4df18b339160af1acdc0'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:50:56.901368 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-1e49d30d-8a42-47b6-bfa7-bc864564df4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57b9a8aa2abd4df18b339160af1acdc0'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:50:56.909704 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-1e49d30d-8a42-47b6-bfa7-bc864564df4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57b9a8aa2abd4df18b339160af1acdc0'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:50:56.928446 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1406/2806] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:50:56 2022] DELETE /identity/v3/users/57b9a8aa2abd4df18b339160af1acdc0 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:50:57.335434 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1420f55f-1d74-4b92-9b4d-f06389dbe335 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:50:57.336901 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-1420f55f-1d74-4b92-9b4d-f06389dbe335 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:50:57.349652 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1420f55f-1d74-4b92-9b4d-f06389dbe335 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:50:57.383425 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:50:57.406455 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694a03f4-d6c8-4a4a-ad58-bd6574e447d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:50:57.407091 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694a03f4-d6c8-4a4a-ad58-bd6574e447d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:50:57.407375 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694a03f4-d6c8-4a4a-ad58-bd6574e447d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:50:57.407656 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694a03f4-d6c8-4a4a-ad58-bd6574e447d6 admin admin] PATH_INFO: `/v3/projects/99843c7e351145f8bfa979be14a15d70` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:50:57.408777 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694a03f4-d6c8-4a4a-ad58-bd6574e447d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99843c7e351145f8bfa979be14a15d70)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:50:57.410734 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694a03f4-d6c8-4a4a-ad58-bd6574e447d6 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:50:57.540428 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-694a03f4-d6c8-4a4a-ad58-bd6574e447d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99843c7e351145f8bfa979be14a15d70', 'request_id': 'req-694a03f4-d6c8-4a4a-ad58-bd6574e447d6'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:50:57.546648 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-694a03f4-d6c8-4a4a-ad58-bd6574e447d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99843c7e351145f8bfa979be14a15d70', 'request_id': 'req-694a03f4-d6c8-4a4a-ad58-bd6574e447d6'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:50:57.626613 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-694a03f4-d6c8-4a4a-ad58-bd6574e447d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99843c7e351145f8bfa979be14a15d70', 'request_id': 'req-694a03f4-d6c8-4a4a-ad58-bd6574e447d6'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:50:57.638783 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-694a03f4-d6c8-4a4a-ad58-bd6574e447d6 admin admin] The token cache is being invalidate because project 99843c7e351145f8bfa979be14a15d70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:50:57.639422 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-694a03f4-d6c8-4a4a-ad58-bd6574e447d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:50:57.640057 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-694a03f4-d6c8-4a4a-ad58-bd6574e447d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:50:57.641741 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1401/2807] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:50:57 2022] DELETE /identity/v3/projects/99843c7e351145f8bfa979be14a15d70 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:50:57.647294 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdf3262-d70f-4d17-8470-876adf535ece None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:50:57.652811 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdf3262-d70f-4d17-8470-876adf535ece None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:50:57.652811 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdf3262-d70f-4d17-8470-876adf535ece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:50:57.667504 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-abdf3262-d70f-4d17-8470-876adf535ece None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:50:57.668704 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-abdf3262-d70f-4d17-8470-876adf535ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:50:57.731353 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1407/2808] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:50:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:50:57.739432 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fd24c4-03ee-4355-b446-f8e359caec9d None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:50:57.739722 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fd24c4-03ee-4355-b446-f8e359caec9d None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:50:57.739990 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fd24c4-03ee-4355-b446-f8e359caec9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:50:57.762677 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-a5fd24c4-03ee-4355-b446-f8e359caec9d None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:50:57.763879 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-a5fd24c4-03ee-4355-b446-f8e359caec9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:50:57.772119 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1402/2809] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:50:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:50:57.776419 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdf3262-d70f-4d17-8470-876adf535ece None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:50:57.786715 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdf3262-d70f-4d17-8470-876adf535ece None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:50:57.792197 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590c1370-d977-4990-92b9-3d37950326dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:50:57.792803 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590c1370-d977-4990-92b9-3d37950326dc admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:50:57.793085 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590c1370-d977-4990-92b9-3d37950326dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:50:57.793357 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590c1370-d977-4990-92b9-3d37950326dc admin admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:50:57.794031 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590c1370-d977-4990-92b9-3d37950326dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:50:57.794333 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590c1370-d977-4990-92b9-3d37950326dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:50:57.795916 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590c1370-d977-4990-92b9-3d37950326dc admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:50:57.808541 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1408/2810] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 15 07:50:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:50:57.812820 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fd24c4-03ee-4355-b446-f8e359caec9d None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:50:57.822375 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fd24c4-03ee-4355-b446-f8e359caec9d None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:50:57.827026 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c56d4d9-d3aa-4c99-91c2-2bf4fde2f2eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:50:57.827632 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c56d4d9-d3aa-4c99-91c2-2bf4fde2f2eb admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:50:57.827915 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c56d4d9-d3aa-4c99-91c2-2bf4fde2f2eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:50:57.828187 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c56d4d9-d3aa-4c99-91c2-2bf4fde2f2eb admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:50:57.828813 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c56d4d9-d3aa-4c99-91c2-2bf4fde2f2eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:50:57.830725 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c56d4d9-d3aa-4c99-91c2-2bf4fde2f2eb admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:50:57.929851 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1403/2811] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:50:57 2022] POST /identity/v3/projects => generated 401 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:50:57.933937 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590c1370-d977-4990-92b9-3d37950326dc admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:50:57.944903 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590c1370-d977-4990-92b9-3d37950326dc admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:50:58.004274 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb142f4-32cb-4917-a89c-c959ff58afc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:50:58.004879 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb142f4-32cb-4917-a89c-c959ff58afc9 admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:50:58.005161 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb142f4-32cb-4917-a89c-c959ff58afc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:50:58.005432 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb142f4-32cb-4917-a89c-c959ff58afc9 admin admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:50:58.006259 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb142f4-32cb-4917-a89c-c959ff58afc9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:50:58.008412 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb142f4-32cb-4917-a89c-c959ff58afc9 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:50:58.019109 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:50:58.088201 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1409/2812] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:50:57 2022] POST /identity/v3/users => generated 301 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:50:58.092473 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c56d4d9-d3aa-4c99-91c2-2bf4fde2f2eb admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:50:58.103396 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c56d4d9-d3aa-4c99-91c2-2bf4fde2f2eb admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:50:58.107878 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d61281-5e60-453c-9550-652514a7c77a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:50:58.108468 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d61281-5e60-453c-9550-652514a7c77a admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:50:58.108749 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d61281-5e60-453c-9550-652514a7c77a admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:50:58.109020 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d61281-5e60-453c-9550-652514a7c77a admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:50:58.109686 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d61281-5e60-453c-9550-652514a7c77a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:50:58.111262 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d61281-5e60-453c-9550-652514a7c77a admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:50:58.121539 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1404/2813] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:50:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:50:58.125667 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb142f4-32cb-4917-a89c-c959ff58afc9 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:50:58.135925 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb142f4-32cb-4917-a89c-c959ff58afc9 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:50:58.140741 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9172bb2-7b75-4133-9910-cc44852fbd4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:50:58.141356 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9172bb2-7b75-4133-9910-cc44852fbd4c admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:50:58.141671 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9172bb2-7b75-4133-9910-cc44852fbd4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:50:58.141951 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9172bb2-7b75-4133-9910-cc44852fbd4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:50:58.142592 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9172bb2-7b75-4133-9910-cc44852fbd4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:50:58.144879 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9172bb2-7b75-4133-9910-cc44852fbd4c admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:50:58.156782 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1410/2814] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:50:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:50:58.160520 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d61281-5e60-453c-9550-652514a7c77a admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:50:58.171292 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d61281-5e60-453c-9550-652514a7c77a admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:50:58.175912 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b581fad7-6368-4884-9c50-751e42259307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:50:58.176564 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b581fad7-6368-4884-9c50-751e42259307 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:50:58.176844 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b581fad7-6368-4884-9c50-751e42259307 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:50:58.177117 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b581fad7-6368-4884-9c50-751e42259307 admin admin] PATH_INFO: `/v3/projects/20c560e30bba4179893dad5bfc02b808/users/333d59fe0518490aa025b28232e36a8e/roles/053e12af84e943e699460a9d76c2072b` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:50:58.192145 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b581fad7-6368-4884-9c50-751e42259307 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20c560e30bba4179893dad5bfc02b808, user_id=333d59fe0518490aa025b28232e36a8e, role_id=053e12af84e943e699460a9d76c2072b)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:50:58.194788 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b581fad7-6368-4884-9c50-751e42259307 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:50:58.248684 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1405/2815] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:50:58 2022] PUT /identity/v3/projects/20c560e30bba4179893dad5bfc02b808/users/333d59fe0518490aa025b28232e36a8e/roles/053e12af84e943e699460a9d76c2072b => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 07:50:58.260637 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f313b9-6308-4681-9d6f-a916052b5284 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:50:58.260924 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f313b9-6308-4681-9d6f-a916052b5284 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:50:58.261200 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f313b9-6308-4681-9d6f-a916052b5284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:50:58.292131 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-c9f313b9-6308-4681-9d6f-a916052b5284 None None] MFA Rules not processed for user `333d59fe0518490aa025b28232e36a8e`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:50:58.293328 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-c9f313b9-6308-4681-9d6f-a916052b5284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:50:58.355585 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1411/2816] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:50:58 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:50:58.360119 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b581fad7-6368-4884-9c50-751e42259307 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:50:58.421422 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b581fad7-6368-4884-9c50-751e42259307 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:50:58.426161 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1987b4c8-497e-4a23-bc7e-392130da5943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:50:58.426763 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1987b4c8-497e-4a23-bc7e-392130da5943 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:50:58.427046 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1987b4c8-497e-4a23-bc7e-392130da5943 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:50:58.427349 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1987b4c8-497e-4a23-bc7e-392130da5943 admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:50:58.428276 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1987b4c8-497e-4a23-bc7e-392130da5943 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:50:58.430272 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1987b4c8-497e-4a23-bc7e-392130da5943 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:50:58.439377 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:50:58.513172 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1406/2817] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:50:58 2022] POST /identity/v3/projects => generated 399 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:50:58.517267 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f313b9-6308-4681-9d6f-a916052b5284 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:50:58.528652 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f313b9-6308-4681-9d6f-a916052b5284 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:50:58.600612 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175dadc7-a46a-4c6e-bb23-08a8520b4118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:50:58.601771 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175dadc7-a46a-4c6e-bb23-08a8520b4118 admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:50:58.601771 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175dadc7-a46a-4c6e-bb23-08a8520b4118 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:50:58.601771 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175dadc7-a46a-4c6e-bb23-08a8520b4118 admin admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:50:58.603002 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175dadc7-a46a-4c6e-bb23-08a8520b4118 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:50:58.604322 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175dadc7-a46a-4c6e-bb23-08a8520b4118 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:50:58.683581 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1412/2818] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:50:58 2022] POST /identity/v3/users => generated 306 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 07:50:58.687912 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1987b4c8-497e-4a23-bc7e-392130da5943 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:50:58.699271 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1987b4c8-497e-4a23-bc7e-392130da5943 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:50:58.703908 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614ba499-c8d2-4411-87e4-3d83a42789b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:50:58.704599 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614ba499-c8d2-4411-87e4-3d83a42789b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:50:58.707264 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614ba499-c8d2-4411-87e4-3d83a42789b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:50:58.707264 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614ba499-c8d2-4411-87e4-3d83a42789b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:50:58.707264 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614ba499-c8d2-4411-87e4-3d83a42789b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:50:58.707567 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614ba499-c8d2-4411-87e4-3d83a42789b3 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:50:58.718081 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1407/2819] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:50:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:50:58.722345 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175dadc7-a46a-4c6e-bb23-08a8520b4118 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:50:58.732220 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175dadc7-a46a-4c6e-bb23-08a8520b4118 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:50:58.736734 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e356f8-d175-4e3c-99d9-a5cf80e06e44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:50:58.737376 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e356f8-d175-4e3c-99d9-a5cf80e06e44 admin admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:50:58.737680 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e356f8-d175-4e3c-99d9-a5cf80e06e44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:50:58.737954 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e356f8-d175-4e3c-99d9-a5cf80e06e44 admin admin] PATH_INFO: `/v3/projects/ca5d570553464809bb3c5c61cb095508/users/eea604c12d9443bb8dec28991f72d74b/roles/6bbb643e0d8d4bc0a9b27c26edf59216` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:50:58.753147 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e356f8-d175-4e3c-99d9-a5cf80e06e44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca5d570553464809bb3c5c61cb095508, user_id=eea604c12d9443bb8dec28991f72d74b, role_id=6bbb643e0d8d4bc0a9b27c26edf59216)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:50:58.755834 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e356f8-d175-4e3c-99d9-a5cf80e06e44 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:50:58.812900 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1413/2820] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:50:58 2022] PUT /identity/v3/projects/ca5d570553464809bb3c5c61cb095508/users/eea604c12d9443bb8dec28991f72d74b/roles/6bbb643e0d8d4bc0a9b27c26edf59216 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 07:50:58.817053 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614ba499-c8d2-4411-87e4-3d83a42789b3 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:50:58.827976 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614ba499-c8d2-4411-87e4-3d83a42789b3 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:50:58.887499 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb0521e-44db-4db4-9941-3cb2167699f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:50:58.888135 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb0521e-44db-4db4-9941-3cb2167699f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:50:58.888416 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb0521e-44db-4db4-9941-3cb2167699f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:50:58.888689 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb0521e-44db-4db4-9941-3cb2167699f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:50:58.889329 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb0521e-44db-4db4-9941-3cb2167699f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:50:58.891359 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb0521e-44db-4db4-9941-3cb2167699f2 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:50:58.901821 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1408/2821] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:50:58 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:50:58.906300 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e356f8-d175-4e3c-99d9-a5cf80e06e44 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:50:58.916726 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e356f8-d175-4e3c-99d9-a5cf80e06e44 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:50:58.921299 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7038b65d-4ce4-4f3f-b2f3-c04faf18697a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:50:58.921982 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7038b65d-4ce4-4f3f-b2f3-c04faf18697a admin admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:50:58.922258 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7038b65d-4ce4-4f3f-b2f3-c04faf18697a admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:50:58.922528 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7038b65d-4ce4-4f3f-b2f3-c04faf18697a admin admin] PATH_INFO: `/v3/projects/ca5d570553464809bb3c5c61cb095508/users/eea604c12d9443bb8dec28991f72d74b/roles/6bbb643e0d8d4bc0a9b27c26edf59216` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:50:58.924390 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7038b65d-4ce4-4f3f-b2f3-c04faf18697a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca5d570553464809bb3c5c61cb095508, user_id=eea604c12d9443bb8dec28991f72d74b, role_id=6bbb643e0d8d4bc0a9b27c26edf59216)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:50:58.927065 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7038b65d-4ce4-4f3f-b2f3-c04faf18697a admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:50:58.937555 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1414/2822] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:50:58 2022] PUT /identity/v3/projects/ca5d570553464809bb3c5c61cb095508/users/eea604c12d9443bb8dec28991f72d74b/roles/6bbb643e0d8d4bc0a9b27c26edf59216 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:50:59.005269 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710251ef-7801-475c-9ed9-e6e93e311170 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:50:59.005723 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710251ef-7801-475c-9ed9-e6e93e311170 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:50:59.010745 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710251ef-7801-475c-9ed9-e6e93e311170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:50:59.034586 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-710251ef-7801-475c-9ed9-e6e93e311170 None None] MFA Rules not processed for user `eea604c12d9443bb8dec28991f72d74b`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:50:59.035852 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-710251ef-7801-475c-9ed9-e6e93e311170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:50:59.103545 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1409/2823] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:50:59 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:50:59.114114 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7038b65d-4ce4-4f3f-b2f3-c04faf18697a admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:50:59.116405 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-7038b65d-4ce4-4f3f-b2f3-c04faf18697a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:50:59.146693 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:50:59.173785 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7038b65d-4ce4-4f3f-b2f3-c04faf18697a admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:50:59.177439 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecb8186-7539-44f9-acec-ea7aa7a2129e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecf6642b7b84d3c9756bb41ab3349d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:50:59.178986 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecb8186-7539-44f9-acec-ea7aa7a2129e service cinder] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:50:59.179512 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecb8186-7539-44f9-acec-ea7aa7a2129e service cinder] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:50:59.180248 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecb8186-7539-44f9-acec-ea7aa7a2129e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:50:59.192037 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecb8186-7539-44f9-acec-ea7aa7a2129e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:50:59.193486 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecb8186-7539-44f9-acec-ea7aa7a2129e service cinder] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:50:59.208043 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1415/2824] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:50:59 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 07:50:59.394608 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710251ef-7801-475c-9ed9-e6e93e311170 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:50:59.404476 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710251ef-7801-475c-9ed9-e6e93e311170 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:50:59.407996 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f881afe-ab50-4456-8ba1-7c775fbe9d99 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecf6642b7b84d3c9756bb41ab3349d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:50:59.408587 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f881afe-ab50-4456-8ba1-7c775fbe9d99 service cinder] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:50:59.413083 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f881afe-ab50-4456-8ba1-7c775fbe9d99 service cinder] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:50:59.413083 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f881afe-ab50-4456-8ba1-7c775fbe9d99 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:50:59.465111 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f881afe-ab50-4456-8ba1-7c775fbe9d99 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:50:59.466550 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f881afe-ab50-4456-8ba1-7c775fbe9d99 service cinder] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:50:59.469986 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:50:59.475984 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1410/2825] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:50:59 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:51:04.468539 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecb8186-7539-44f9-acec-ea7aa7a2129e service cinder] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:51:04.530945 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecb8186-7539-44f9-acec-ea7aa7a2129e service cinder] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:51:04.535414 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282a1971-470c-4ca6-a356-77ed6304b479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:51:04.536043 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282a1971-470c-4ca6-a356-77ed6304b479 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:04.536324 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282a1971-470c-4ca6-a356-77ed6304b479 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:04.536601 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282a1971-470c-4ca6-a356-77ed6304b479 admin admin] PATH_INFO: `/v3/users/333d59fe0518490aa025b28232e36a8e` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:04.537744 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282a1971-470c-4ca6-a356-77ed6304b479 admin admin] RBAC: Authorizing `identity:delete_user(user_id=333d59fe0518490aa025b28232e36a8e)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:51:04.539653 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282a1971-470c-4ca6-a356-77ed6304b479 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:51:04.730660 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-282a1971-470c-4ca6-a356-77ed6304b479 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '333d59fe0518490aa025b28232e36a8e'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:04.796807 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-282a1971-470c-4ca6-a356-77ed6304b479 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '333d59fe0518490aa025b28232e36a8e'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:04.797439 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-282a1971-470c-4ca6-a356-77ed6304b479 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '333d59fe0518490aa025b28232e36a8e'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:04.798077 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-282a1971-470c-4ca6-a356-77ed6304b479 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '333d59fe0518490aa025b28232e36a8e'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:04.805397 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-282a1971-470c-4ca6-a356-77ed6304b479 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '333d59fe0518490aa025b28232e36a8e'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:04.824070 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1416/2826] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:51:04 2022] DELETE /identity/v3/users/333d59fe0518490aa025b28232e36a8e => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:51:04.828586 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f24bc24-ef72-41db-817e-d32a1177d205 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:04.829078 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f24bc24-ef72-41db-817e-d32a1177d205 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:04.829196 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f24bc24-ef72-41db-817e-d32a1177d205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:04.848356 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-4f24bc24-ef72-41db-817e-d32a1177d205 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:51:04.849543 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-4f24bc24-ef72-41db-817e-d32a1177d205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:51:04.906626 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1411/2827] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:51:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 07:51:04.916812 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282a1971-470c-4ca6-a356-77ed6304b479 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:51:04.918240 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-282a1971-470c-4ca6-a356-77ed6304b479 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:51:04.972956 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:51:04.976724 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282a1971-470c-4ca6-a356-77ed6304b479 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:51:04.981203 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520dacd4-0be8-4e1d-b7e1-73f0d857d20a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:51:04.981927 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520dacd4-0be8-4e1d-b7e1-73f0d857d20a service neutron] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:04.982209 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520dacd4-0be8-4e1d-b7e1-73f0d857d20a service neutron] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:04.984337 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520dacd4-0be8-4e1d-b7e1-73f0d857d20a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:04.991510 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520dacd4-0be8-4e1d-b7e1-73f0d857d20a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:51:04.993001 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520dacd4-0be8-4e1d-b7e1-73f0d857d20a service neutron] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:51:05.003658 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1417/2828] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:51:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:51:05.414721 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f24bc24-ef72-41db-817e-d32a1177d205 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:51:05.418966 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-4f24bc24-ef72-41db-817e-d32a1177d205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:51:05.428221 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f24bc24-ef72-41db-817e-d32a1177d205 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:51:05.433549 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e2133e-c88c-40bf-85bb-0d701ecc0181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:51:05.434230 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e2133e-c88c-40bf-85bb-0d701ecc0181 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:05.434515 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e2133e-c88c-40bf-85bb-0d701ecc0181 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:05.434795 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e2133e-c88c-40bf-85bb-0d701ecc0181 admin admin] PATH_INFO: `/v3/projects/20c560e30bba4179893dad5bfc02b808` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:05.435969 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e2133e-c88c-40bf-85bb-0d701ecc0181 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20c560e30bba4179893dad5bfc02b808)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:51:05.437909 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e2133e-c88c-40bf-85bb-0d701ecc0181 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:51:05.539890 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-68e2133e-c88c-40bf-85bb-0d701ecc0181 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20c560e30bba4179893dad5bfc02b808', 'request_id': 'req-68e2133e-c88c-40bf-85bb-0d701ecc0181'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:05.546103 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-68e2133e-c88c-40bf-85bb-0d701ecc0181 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20c560e30bba4179893dad5bfc02b808', 'request_id': 'req-68e2133e-c88c-40bf-85bb-0d701ecc0181'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:05.602816 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-68e2133e-c88c-40bf-85bb-0d701ecc0181 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20c560e30bba4179893dad5bfc02b808', 'request_id': 'req-68e2133e-c88c-40bf-85bb-0d701ecc0181'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:05.614105 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-68e2133e-c88c-40bf-85bb-0d701ecc0181 admin admin] The token cache is being invalidate because project 20c560e30bba4179893dad5bfc02b808 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:51:05.614398 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-68e2133e-c88c-40bf-85bb-0d701ecc0181 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:05.615031 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-68e2133e-c88c-40bf-85bb-0d701ecc0181 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:05.616567 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1412/2829] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:51:05 2022] DELETE /identity/v3/projects/20c560e30bba4179893dad5bfc02b808 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:51:05.621235 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520dacd4-0be8-4e1d-b7e1-73f0d857d20a service neutron] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:51:05.630979 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520dacd4-0be8-4e1d-b7e1-73f0d857d20a service neutron] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:51:05.688352 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b04e211-2034-4c03-ae20-23222dd02287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:51:05.688940 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b04e211-2034-4c03-ae20-23222dd02287 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:05.689221 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b04e211-2034-4c03-ae20-23222dd02287 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:05.689499 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b04e211-2034-4c03-ae20-23222dd02287 admin admin] PATH_INFO: `/v3/users/eea604c12d9443bb8dec28991f72d74b` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:05.690656 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b04e211-2034-4c03-ae20-23222dd02287 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eea604c12d9443bb8dec28991f72d74b)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:51:05.692680 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b04e211-2034-4c03-ae20-23222dd02287 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:51:05.856181 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-2b04e211-2034-4c03-ae20-23222dd02287 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eea604c12d9443bb8dec28991f72d74b'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:05.932808 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-2b04e211-2034-4c03-ae20-23222dd02287 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eea604c12d9443bb8dec28991f72d74b'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:05.933710 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-2b04e211-2034-4c03-ae20-23222dd02287 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eea604c12d9443bb8dec28991f72d74b'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:05.934382 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-2b04e211-2034-4c03-ae20-23222dd02287 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eea604c12d9443bb8dec28991f72d74b'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:05.944227 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-2b04e211-2034-4c03-ae20-23222dd02287 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eea604c12d9443bb8dec28991f72d74b'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:05.962050 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1418/2830] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:51:05 2022] DELETE /identity/v3/users/eea604c12d9443bb8dec28991f72d74b => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:51:06.375842 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e2133e-c88c-40bf-85bb-0d701ecc0181 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:51:06.377313 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-68e2133e-c88c-40bf-85bb-0d701ecc0181 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:51:06.390422 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e2133e-c88c-40bf-85bb-0d701ecc0181 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:51:06.448623 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128ed953-d1f1-4019-9b20-7f313ec9ce10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:51:06.449264 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128ed953-d1f1-4019-9b20-7f313ec9ce10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:06.449590 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128ed953-d1f1-4019-9b20-7f313ec9ce10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:06.449869 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128ed953-d1f1-4019-9b20-7f313ec9ce10 admin admin] PATH_INFO: `/v3/projects/ca5d570553464809bb3c5c61cb095508` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:06.451012 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128ed953-d1f1-4019-9b20-7f313ec9ce10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca5d570553464809bb3c5c61cb095508)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:51:06.452906 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128ed953-d1f1-4019-9b20-7f313ec9ce10 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:51:06.477073 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:51:06.551626 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-128ed953-d1f1-4019-9b20-7f313ec9ce10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca5d570553464809bb3c5c61cb095508', 'request_id': 'req-128ed953-d1f1-4019-9b20-7f313ec9ce10'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:06.559873 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-128ed953-d1f1-4019-9b20-7f313ec9ce10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca5d570553464809bb3c5c61cb095508', 'request_id': 'req-128ed953-d1f1-4019-9b20-7f313ec9ce10'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:06.613890 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-128ed953-d1f1-4019-9b20-7f313ec9ce10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca5d570553464809bb3c5c61cb095508', 'request_id': 'req-128ed953-d1f1-4019-9b20-7f313ec9ce10'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:06.625044 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-128ed953-d1f1-4019-9b20-7f313ec9ce10 admin admin] The token cache is being invalidate because project ca5d570553464809bb3c5c61cb095508 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:51:06.625331 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-128ed953-d1f1-4019-9b20-7f313ec9ce10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:06.625970 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-128ed953-d1f1-4019-9b20-7f313ec9ce10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:06.627757 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1413/2831] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:51:06 2022] DELETE /identity/v3/projects/ca5d570553464809bb3c5c61cb095508 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:51:06.632659 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f66c216-1960-4b80-86c9-7d17363bffde None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:06.633146 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f66c216-1960-4b80-86c9-7d17363bffde None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:06.633276 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f66c216-1960-4b80-86c9-7d17363bffde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:06.652921 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-3f66c216-1960-4b80-86c9-7d17363bffde None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:51:06.654133 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-3f66c216-1960-4b80-86c9-7d17363bffde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:51:06.714661 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1419/2832] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:51:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:51:06.722995 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f835dca6-143e-4188-9313-f100e134b785 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:06.723282 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f835dca6-143e-4188-9313-f100e134b785 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:06.723558 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f835dca6-143e-4188-9313-f100e134b785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:06.743837 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-f835dca6-143e-4188-9313-f100e134b785 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:51:06.744954 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-f835dca6-143e-4188-9313-f100e134b785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:51:06.752846 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1414/2833] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:51:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:51:06.757037 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f66c216-1960-4b80-86c9-7d17363bffde None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:51:06.767340 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f66c216-1960-4b80-86c9-7d17363bffde None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:51:06.771941 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535108c8-903b-4c6f-b147-ef3beb2512db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:51:06.772536 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535108c8-903b-4c6f-b147-ef3beb2512db admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:06.772820 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535108c8-903b-4c6f-b147-ef3beb2512db admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:06.773090 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535108c8-903b-4c6f-b147-ef3beb2512db admin admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:06.773872 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535108c8-903b-4c6f-b147-ef3beb2512db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:51:06.774176 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535108c8-903b-4c6f-b147-ef3beb2512db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:51:06.775754 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535108c8-903b-4c6f-b147-ef3beb2512db admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:51:06.783509 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:51:06.788414 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1420/2834] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 15 07:51:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:51:06.793419 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f835dca6-143e-4188-9313-f100e134b785 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:51:06.802636 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f835dca6-143e-4188-9313-f100e134b785 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:51:06.807420 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd51181f-3a70-488a-a7cd-0684179e082f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:51:06.808007 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd51181f-3a70-488a-a7cd-0684179e082f admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:06.808285 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd51181f-3a70-488a-a7cd-0684179e082f admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:06.808556 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd51181f-3a70-488a-a7cd-0684179e082f admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:06.809174 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd51181f-3a70-488a-a7cd-0684179e082f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:51:06.811083 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd51181f-3a70-488a-a7cd-0684179e082f admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:51:06.928467 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1415/2835] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:51:06 2022] POST /identity/v3/projects => generated 379 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:51:06.932501 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535108c8-903b-4c6f-b147-ef3beb2512db admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:51:06.942963 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535108c8-903b-4c6f-b147-ef3beb2512db admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:51:06.999216 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccfaa23-5c08-4ef0-8027-e20ac67c39ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:51:06.999738 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccfaa23-5c08-4ef0-8027-e20ac67c39ed admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:07.000017 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccfaa23-5c08-4ef0-8027-e20ac67c39ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:07.000288 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccfaa23-5c08-4ef0-8027-e20ac67c39ed admin admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:07.000905 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccfaa23-5c08-4ef0-8027-e20ac67c39ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:51:07.002830 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccfaa23-5c08-4ef0-8027-e20ac67c39ed admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:51:07.082133 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1421/2836] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:51:06 2022] POST /identity/v3/users => generated 290 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:51:07.086455 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd51181f-3a70-488a-a7cd-0684179e082f admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:51:07.097332 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd51181f-3a70-488a-a7cd-0684179e082f admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:51:07.101914 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd177ff-01b3-448c-afbe-7c7913ffbeff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:51:07.103134 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd177ff-01b3-448c-afbe-7c7913ffbeff admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:07.103134 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd177ff-01b3-448c-afbe-7c7913ffbeff admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:07.103134 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd177ff-01b3-448c-afbe-7c7913ffbeff admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:07.103827 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd177ff-01b3-448c-afbe-7c7913ffbeff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:51:07.106074 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd177ff-01b3-448c-afbe-7c7913ffbeff admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:51:07.116109 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1416/2837] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:51:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:51:07.119931 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccfaa23-5c08-4ef0-8027-e20ac67c39ed admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:51:07.129918 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccfaa23-5c08-4ef0-8027-e20ac67c39ed admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:51:07.136527 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ea4fa8-71d0-49a2-85d7-aa501f3bcfb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:51:07.137544 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ea4fa8-71d0-49a2-85d7-aa501f3bcfb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:07.138037 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ea4fa8-71d0-49a2-85d7-aa501f3bcfb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:07.138466 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ea4fa8-71d0-49a2-85d7-aa501f3bcfb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:07.139506 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ea4fa8-71d0-49a2-85d7-aa501f3bcfb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:51:07.141637 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ea4fa8-71d0-49a2-85d7-aa501f3bcfb9 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:51:07.153406 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1422/2838] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:51:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:51:07.157535 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd177ff-01b3-448c-afbe-7c7913ffbeff admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:51:07.167736 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd177ff-01b3-448c-afbe-7c7913ffbeff admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:51:07.172400 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9775b3-e4e2-4952-aafd-1b7d1e1bab5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:51:07.173078 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9775b3-e4e2-4952-aafd-1b7d1e1bab5f admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:07.173364 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9775b3-e4e2-4952-aafd-1b7d1e1bab5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:07.173673 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9775b3-e4e2-4952-aafd-1b7d1e1bab5f admin admin] PATH_INFO: `/v3/projects/e3900518a6584cab94bffda86cbeb5b7/users/13ab8c4149ed4702a3016b79aecd8665/roles/053e12af84e943e699460a9d76c2072b` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:07.188078 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9775b3-e4e2-4952-aafd-1b7d1e1bab5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3900518a6584cab94bffda86cbeb5b7, user_id=13ab8c4149ed4702a3016b79aecd8665, role_id=053e12af84e943e699460a9d76c2072b)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:51:07.190750 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9775b3-e4e2-4952-aafd-1b7d1e1bab5f admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:51:07.271412 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1417/2839] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:51:07 2022] PUT /identity/v3/projects/e3900518a6584cab94bffda86cbeb5b7/users/13ab8c4149ed4702a3016b79aecd8665/roles/053e12af84e943e699460a9d76c2072b => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:51:07.284026 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c1a9db-e79b-46f0-b066-7c09556b7e2c None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:07.284317 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c1a9db-e79b-46f0-b066-7c09556b7e2c None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:07.284598 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c1a9db-e79b-46f0-b066-7c09556b7e2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:07.312568 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-30c1a9db-e79b-46f0-b066-7c09556b7e2c None None] MFA Rules not processed for user `13ab8c4149ed4702a3016b79aecd8665`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:51:07.315270 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-30c1a9db-e79b-46f0-b066-7c09556b7e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:51:07.378338 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1423/2840] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:51:07 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:51:07.382824 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9775b3-e4e2-4952-aafd-1b7d1e1bab5f admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:51:07.444655 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9775b3-e4e2-4952-aafd-1b7d1e1bab5f admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:51:07.449848 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fd2c2a-7104-4c62-95ea-efdc2249d92b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:51:07.450446 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fd2c2a-7104-4c62-95ea-efdc2249d92b admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:07.450724 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fd2c2a-7104-4c62-95ea-efdc2249d92b admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:07.450994 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fd2c2a-7104-4c62-95ea-efdc2249d92b admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:07.451631 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fd2c2a-7104-4c62-95ea-efdc2249d92b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:51:07.453506 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fd2c2a-7104-4c62-95ea-efdc2249d92b admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:51:07.564670 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1418/2841] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:51:07 2022] POST /identity/v3/projects => generated 377 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:51:07.568824 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c1a9db-e79b-46f0-b066-7c09556b7e2c None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:51:07.580740 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c1a9db-e79b-46f0-b066-7c09556b7e2c None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:51:07.643228 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16a5c60-85b4-4f3a-bf1f-22dd96f9966f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:51:07.643922 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16a5c60-85b4-4f3a-bf1f-22dd96f9966f admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:07.644142 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16a5c60-85b4-4f3a-bf1f-22dd96f9966f admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:07.644373 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16a5c60-85b4-4f3a-bf1f-22dd96f9966f admin admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:07.645020 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16a5c60-85b4-4f3a-bf1f-22dd96f9966f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:51:07.646974 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16a5c60-85b4-4f3a-bf1f-22dd96f9966f admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:51:07.658202 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:51:07.763582 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1424/2842] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:51:07 2022] POST /identity/v3/users => generated 289 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 07:51:07.767544 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fd2c2a-7104-4c62-95ea-efdc2249d92b admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:51:07.779292 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fd2c2a-7104-4c62-95ea-efdc2249d92b admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:51:07.783898 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad23127-cab1-42b5-afae-6d41bf39df4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:51:07.784508 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad23127-cab1-42b5-afae-6d41bf39df4d admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:07.784791 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad23127-cab1-42b5-afae-6d41bf39df4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:07.785068 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad23127-cab1-42b5-afae-6d41bf39df4d admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:07.785768 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad23127-cab1-42b5-afae-6d41bf39df4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:51:07.787367 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad23127-cab1-42b5-afae-6d41bf39df4d admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:51:07.798672 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1419/2843] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:51:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:51:07.802593 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16a5c60-85b4-4f3a-bf1f-22dd96f9966f admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:51:07.812403 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16a5c60-85b4-4f3a-bf1f-22dd96f9966f admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:51:07.817186 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe354ae-559a-467e-aad2-9638ea19ad1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:51:07.817758 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe354ae-559a-467e-aad2-9638ea19ad1d admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:07.818034 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe354ae-559a-467e-aad2-9638ea19ad1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:07.818307 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe354ae-559a-467e-aad2-9638ea19ad1d admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:07.818937 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe354ae-559a-467e-aad2-9638ea19ad1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:51:07.820513 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe354ae-559a-467e-aad2-9638ea19ad1d admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:51:07.830622 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1425/2844] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:51:07 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:51:07.834874 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad23127-cab1-42b5-afae-6d41bf39df4d admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:51:07.845258 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad23127-cab1-42b5-afae-6d41bf39df4d admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:51:07.849821 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca18b508-7ddf-4318-8f6d-46f242424c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:51:07.850475 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca18b508-7ddf-4318-8f6d-46f242424c33 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:07.850760 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca18b508-7ddf-4318-8f6d-46f242424c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:07.851039 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca18b508-7ddf-4318-8f6d-46f242424c33 admin admin] PATH_INFO: `/v3/projects/57d91c68cb8542b598108e6bf4152800/users/792ea6b1a91b4241a83cc92f122a6485/roles/053e12af84e943e699460a9d76c2072b` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:07.865400 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca18b508-7ddf-4318-8f6d-46f242424c33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57d91c68cb8542b598108e6bf4152800, user_id=792ea6b1a91b4241a83cc92f122a6485, role_id=053e12af84e943e699460a9d76c2072b)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:51:07.868063 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca18b508-7ddf-4318-8f6d-46f242424c33 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:51:07.911590 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=89,keystone:DELETE=6,keystone:INSERT=5 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:51:07.916878 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1420/2845] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:51:07 2022] PUT /identity/v3/projects/57d91c68cb8542b598108e6bf4152800/users/792ea6b1a91b4241a83cc92f122a6485/roles/053e12af84e943e699460a9d76c2072b => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:51:07.927616 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4243c7-d1a1-4b28-bc83-2e9e0c072ac2 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:07.927903 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4243c7-d1a1-4b28-bc83-2e9e0c072ac2 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:07.928182 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4243c7-d1a1-4b28-bc83-2e9e0c072ac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:07.958142 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-4e4243c7-d1a1-4b28-bc83-2e9e0c072ac2 None None] MFA Rules not processed for user `792ea6b1a91b4241a83cc92f122a6485`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:51:07.959668 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-4e4243c7-d1a1-4b28-bc83-2e9e0c072ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:51:08.027807 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1426/2846] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:51:07 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:51:08.032318 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca18b508-7ddf-4318-8f6d-46f242424c33 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:51:08.093485 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca18b508-7ddf-4318-8f6d-46f242424c33 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:51:08.097999 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507c991b-f889-44b0-8166-f0c659f3aaae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:51:08.098597 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507c991b-f889-44b0-8166-f0c659f3aaae admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:08.098875 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507c991b-f889-44b0-8166-f0c659f3aaae admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:08.099148 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507c991b-f889-44b0-8166-f0c659f3aaae admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:08.099767 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507c991b-f889-44b0-8166-f0c659f3aaae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:51:08.101655 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507c991b-f889-44b0-8166-f0c659f3aaae admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:51:08.205496 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1421/2847] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:51:08 2022] POST /identity/v3/projects => generated 377 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:51:08.213282 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4243c7-d1a1-4b28-bc83-2e9e0c072ac2 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:51:08.224638 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4243c7-d1a1-4b28-bc83-2e9e0c072ac2 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:51:08.279867 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa735d2-1206-4ae5-bec4-106c332aa885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:51:08.280476 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa735d2-1206-4ae5-bec4-106c332aa885 admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:08.280755 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa735d2-1206-4ae5-bec4-106c332aa885 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:08.281026 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa735d2-1206-4ae5-bec4-106c332aa885 admin admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:08.281676 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa735d2-1206-4ae5-bec4-106c332aa885 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:51:08.283579 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa735d2-1206-4ae5-bec4-106c332aa885 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:51:08.372050 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1427/2848] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:51:08 2022] POST /identity/v3/users => generated 295 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:51:08.376020 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507c991b-f889-44b0-8166-f0c659f3aaae admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:51:08.387519 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507c991b-f889-44b0-8166-f0c659f3aaae admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:51:08.392265 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeb2973-7d98-45d1-9c9b-15fed1a675ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:51:08.392868 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeb2973-7d98-45d1-9c9b-15fed1a675ab admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:08.393250 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeb2973-7d98-45d1-9c9b-15fed1a675ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:08.393534 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeb2973-7d98-45d1-9c9b-15fed1a675ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:08.394214 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeb2973-7d98-45d1-9c9b-15fed1a675ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:51:08.395905 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeb2973-7d98-45d1-9c9b-15fed1a675ab admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:51:08.406679 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1422/2849] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:51:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:51:08.410904 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa735d2-1206-4ae5-bec4-106c332aa885 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:51:08.420861 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa735d2-1206-4ae5-bec4-106c332aa885 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:51:08.425342 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04b98e5-f652-4c50-a6b3-b6a4f53b2aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:51:08.426018 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b98e5-f652-4c50-a6b3-b6a4f53b2aa9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:08.426296 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b98e5-f652-4c50-a6b3-b6a4f53b2aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:08.426567 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b98e5-f652-4c50-a6b3-b6a4f53b2aa9 admin admin] PATH_INFO: `/v3/projects/1a8aa7d71de44cb7b2b1821fbd4f3c77/users/2e404445f35a4d23a5dda5d171a66e9f/roles/6bbb643e0d8d4bc0a9b27c26edf59216` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:08.441482 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04b98e5-f652-4c50-a6b3-b6a4f53b2aa9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a8aa7d71de44cb7b2b1821fbd4f3c77, user_id=2e404445f35a4d23a5dda5d171a66e9f, role_id=6bbb643e0d8d4bc0a9b27c26edf59216)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:51:08.444133 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04b98e5-f652-4c50-a6b3-b6a4f53b2aa9 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:51:08.509851 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1428/2850] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:51:08 2022] PUT /identity/v3/projects/1a8aa7d71de44cb7b2b1821fbd4f3c77/users/2e404445f35a4d23a5dda5d171a66e9f/roles/6bbb643e0d8d4bc0a9b27c26edf59216 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:51:08.514464 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeb2973-7d98-45d1-9c9b-15fed1a675ab admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:51:08.524483 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeb2973-7d98-45d1-9c9b-15fed1a675ab admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:51:08.591172 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2559622-bcf3-4475-ba8c-613c2dadd14c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:51:08.591770 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2559622-bcf3-4475-ba8c-613c2dadd14c admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:08.592049 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2559622-bcf3-4475-ba8c-613c2dadd14c admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:08.592321 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2559622-bcf3-4475-ba8c-613c2dadd14c admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:08.592951 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2559622-bcf3-4475-ba8c-613c2dadd14c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:51:08.594553 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2559622-bcf3-4475-ba8c-613c2dadd14c admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:51:08.604072 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1423/2851] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:51:08 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:51:08.609379 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04b98e5-f652-4c50-a6b3-b6a4f53b2aa9 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:51:08.619619 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04b98e5-f652-4c50-a6b3-b6a4f53b2aa9 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:51:08.625343 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1e3ec4-736a-431c-98f9-fc65731e1092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:51:08.625948 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1e3ec4-736a-431c-98f9-fc65731e1092 admin admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:08.626230 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1e3ec4-736a-431c-98f9-fc65731e1092 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:08.626636 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1e3ec4-736a-431c-98f9-fc65731e1092 admin admin] PATH_INFO: `/v3/projects/1a8aa7d71de44cb7b2b1821fbd4f3c77/users/2e404445f35a4d23a5dda5d171a66e9f/roles/6bbb643e0d8d4bc0a9b27c26edf59216` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:08.628548 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1e3ec4-736a-431c-98f9-fc65731e1092 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a8aa7d71de44cb7b2b1821fbd4f3c77, user_id=2e404445f35a4d23a5dda5d171a66e9f, role_id=6bbb643e0d8d4bc0a9b27c26edf59216)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:51:08.631228 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1e3ec4-736a-431c-98f9-fc65731e1092 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:51:08.642248 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1429/2852] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:51:08 2022] PUT /identity/v3/projects/1a8aa7d71de44cb7b2b1821fbd4f3c77/users/2e404445f35a4d23a5dda5d171a66e9f/roles/6bbb643e0d8d4bc0a9b27c26edf59216 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:51:08.659970 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf399b9-e545-4de6-bd50-30fa7d6f841d None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:08.660274 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf399b9-e545-4de6-bd50-30fa7d6f841d None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:08.660587 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf399b9-e545-4de6-bd50-30fa7d6f841d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:08.690697 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-ddf399b9-e545-4de6-bd50-30fa7d6f841d None None] MFA Rules not processed for user `2e404445f35a4d23a5dda5d171a66e9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:51:08.691906 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-ddf399b9-e545-4de6-bd50-30fa7d6f841d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:51:08.751385 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:51:08.761257 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1424/2853] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:51:08 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:51:08.775127 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1e3ec4-736a-431c-98f9-fc65731e1092 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:51:08.777158 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-4c1e3ec4-736a-431c-98f9-fc65731e1092 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:51:08.809101 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:51:08.835734 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1e3ec4-736a-431c-98f9-fc65731e1092 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:51:08.840038 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e1f0b2-18e9-41c3-9bd1-ad6d5db2fe7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecf6642b7b84d3c9756bb41ab3349d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:51:08.840637 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e1f0b2-18e9-41c3-9bd1-ad6d5db2fe7b service cinder] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:08.840915 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e1f0b2-18e9-41c3-9bd1-ad6d5db2fe7b service cinder] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:08.841185 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e1f0b2-18e9-41c3-9bd1-ad6d5db2fe7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:08.850171 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e1f0b2-18e9-41c3-9bd1-ad6d5db2fe7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:51:08.851635 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e1f0b2-18e9-41c3-9bd1-ad6d5db2fe7b service cinder] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:51:08.861424 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1430/2854] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:51:08 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:51:08.993315 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf399b9-e545-4de6-bd50-30fa7d6f841d None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:51:08.994790 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-ddf399b9-e545-4de6-bd50-30fa7d6f841d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:51:09.007237 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf399b9-e545-4de6-bd50-30fa7d6f841d None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:51:09.010936 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54576e9f-1a89-4d09-93ab-94ab34f2f52a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecf6642b7b84d3c9756bb41ab3349d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:51:09.011682 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54576e9f-1a89-4d09-93ab-94ab34f2f52a service cinder] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:09.011957 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54576e9f-1a89-4d09-93ab-94ab34f2f52a service cinder] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:09.012228 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54576e9f-1a89-4d09-93ab-94ab34f2f52a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:09.068295 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54576e9f-1a89-4d09-93ab-94ab34f2f52a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:51:09.069727 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54576e9f-1a89-4d09-93ab-94ab34f2f52a service cinder] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:51:09.078890 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1425/2855] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:51:08 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:51:13.051025 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e1f0b2-18e9-41c3-9bd1-ad6d5db2fe7b service cinder] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:51:13.112456 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e1f0b2-18e9-41c3-9bd1-ad6d5db2fe7b service cinder] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:51:13.117032 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adcd451-8844-4863-99b9-a653aa6751a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:51:13.119494 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adcd451-8844-4863-99b9-a653aa6751a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:13.119494 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adcd451-8844-4863-99b9-a653aa6751a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:13.119494 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adcd451-8844-4863-99b9-a653aa6751a0 admin admin] PATH_INFO: `/v3/users/13ab8c4149ed4702a3016b79aecd8665` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:13.119494 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adcd451-8844-4863-99b9-a653aa6751a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13ab8c4149ed4702a3016b79aecd8665)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:51:13.121294 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adcd451-8844-4863-99b9-a653aa6751a0 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:51:13.276823 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-7adcd451-8844-4863-99b9-a653aa6751a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13ab8c4149ed4702a3016b79aecd8665'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:13.350807 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-7adcd451-8844-4863-99b9-a653aa6751a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13ab8c4149ed4702a3016b79aecd8665'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:13.351471 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-7adcd451-8844-4863-99b9-a653aa6751a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13ab8c4149ed4702a3016b79aecd8665'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:13.352102 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-7adcd451-8844-4863-99b9-a653aa6751a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13ab8c4149ed4702a3016b79aecd8665'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:13.359729 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-7adcd451-8844-4863-99b9-a653aa6751a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13ab8c4149ed4702a3016b79aecd8665'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:13.376979 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1431/2856] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:51:13 2022] DELETE /identity/v3/users/13ab8c4149ed4702a3016b79aecd8665 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:51:13.381473 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9ca854-3044-4965-a592-13e6e3d6df70 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:13.382000 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9ca854-3044-4965-a592-13e6e3d6df70 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:13.382116 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9ca854-3044-4965-a592-13e6e3d6df70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:13.401476 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-fc9ca854-3044-4965-a592-13e6e3d6df70 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:51:13.403355 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-fc9ca854-3044-4965-a592-13e6e3d6df70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:51:13.461319 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1426/2857] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:51:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:51:13.471162 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adcd451-8844-4863-99b9-a653aa6751a0 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:51:13.472556 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-7adcd451-8844-4863-99b9-a653aa6751a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:51:13.526731 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:51:13.530674 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adcd451-8844-4863-99b9-a653aa6751a0 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:51:13.534339 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1be99af-9d9a-4643-a12d-2b36484cdeb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:51:13.534946 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be99af-9d9a-4643-a12d-2b36484cdeb8 service neutron] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:13.535231 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be99af-9d9a-4643-a12d-2b36484cdeb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:13.535509 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be99af-9d9a-4643-a12d-2b36484cdeb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:13.544542 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1be99af-9d9a-4643-a12d-2b36484cdeb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:51:13.546023 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1be99af-9d9a-4643-a12d-2b36484cdeb8 service neutron] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:51:13.563463 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1432/2858] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:51:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:51:13.963833 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9ca854-3044-4965-a592-13e6e3d6df70 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:51:13.965825 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-fc9ca854-3044-4965-a592-13e6e3d6df70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:51:13.977977 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9ca854-3044-4965-a592-13e6e3d6df70 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:51:13.983671 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5484b55-d20a-4864-9deb-e6a7b0672d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:51:13.984334 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5484b55-d20a-4864-9deb-e6a7b0672d05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:13.984615 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5484b55-d20a-4864-9deb-e6a7b0672d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:13.984886 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5484b55-d20a-4864-9deb-e6a7b0672d05 admin admin] PATH_INFO: `/v3/projects/e3900518a6584cab94bffda86cbeb5b7` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:13.986057 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5484b55-d20a-4864-9deb-e6a7b0672d05 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3900518a6584cab94bffda86cbeb5b7)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:51:13.990652 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5484b55-d20a-4864-9deb-e6a7b0672d05 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:51:14.111421 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-d5484b55-d20a-4864-9deb-e6a7b0672d05 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3900518a6584cab94bffda86cbeb5b7', 'request_id': 'req-d5484b55-d20a-4864-9deb-e6a7b0672d05'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:14.117681 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-d5484b55-d20a-4864-9deb-e6a7b0672d05 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3900518a6584cab94bffda86cbeb5b7', 'request_id': 'req-d5484b55-d20a-4864-9deb-e6a7b0672d05'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:14.179889 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-d5484b55-d20a-4864-9deb-e6a7b0672d05 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3900518a6584cab94bffda86cbeb5b7', 'request_id': 'req-d5484b55-d20a-4864-9deb-e6a7b0672d05'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:14.193072 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-d5484b55-d20a-4864-9deb-e6a7b0672d05 admin admin] The token cache is being invalidate because project e3900518a6584cab94bffda86cbeb5b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:51:14.193376 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-d5484b55-d20a-4864-9deb-e6a7b0672d05 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:14.194091 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-d5484b55-d20a-4864-9deb-e6a7b0672d05 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:14.197046 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1427/2859] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:51:13 2022] DELETE /identity/v3/projects/e3900518a6584cab94bffda86cbeb5b7 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:51:14.202150 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1be99af-9d9a-4643-a12d-2b36484cdeb8 service neutron] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:51:14.212440 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1be99af-9d9a-4643-a12d-2b36484cdeb8 service neutron] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:51:14.268499 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fee8d0a-6227-442f-bf0e-752850ca8869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:51:14.269126 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fee8d0a-6227-442f-bf0e-752850ca8869 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:14.269407 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fee8d0a-6227-442f-bf0e-752850ca8869 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:14.269710 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fee8d0a-6227-442f-bf0e-752850ca8869 admin admin] PATH_INFO: `/v3/users/792ea6b1a91b4241a83cc92f122a6485` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:14.270817 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fee8d0a-6227-442f-bf0e-752850ca8869 admin admin] RBAC: Authorizing `identity:delete_user(user_id=792ea6b1a91b4241a83cc92f122a6485)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:51:14.272732 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fee8d0a-6227-442f-bf0e-752850ca8869 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:51:14.416885 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-7fee8d0a-6227-442f-bf0e-752850ca8869 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '792ea6b1a91b4241a83cc92f122a6485'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:14.482631 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-7fee8d0a-6227-442f-bf0e-752850ca8869 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '792ea6b1a91b4241a83cc92f122a6485'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:14.483198 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-7fee8d0a-6227-442f-bf0e-752850ca8869 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '792ea6b1a91b4241a83cc92f122a6485'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:14.483795 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-7fee8d0a-6227-442f-bf0e-752850ca8869 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '792ea6b1a91b4241a83cc92f122a6485'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:14.491537 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-7fee8d0a-6227-442f-bf0e-752850ca8869 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '792ea6b1a91b4241a83cc92f122a6485'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:14.510115 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1433/2860] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:51:14 2022] DELETE /identity/v3/users/792ea6b1a91b4241a83cc92f122a6485 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:51:14.906711 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5484b55-d20a-4864-9deb-e6a7b0672d05 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:51:14.908157 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-d5484b55-d20a-4864-9deb-e6a7b0672d05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:51:14.922797 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5484b55-d20a-4864-9deb-e6a7b0672d05 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:51:14.937483 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:51:14.983425 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cdeafd-ce46-484b-a00a-51e644fceacc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:51:14.984087 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cdeafd-ce46-484b-a00a-51e644fceacc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:14.984578 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cdeafd-ce46-484b-a00a-51e644fceacc admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:14.985019 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cdeafd-ce46-484b-a00a-51e644fceacc admin admin] PATH_INFO: `/v3/projects/57d91c68cb8542b598108e6bf4152800` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:14.986524 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cdeafd-ce46-484b-a00a-51e644fceacc admin admin] RBAC: Authorizing `identity:delete_project(project_id=57d91c68cb8542b598108e6bf4152800)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:51:14.988417 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cdeafd-ce46-484b-a00a-51e644fceacc admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:51:15.082013 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-15cdeafd-ce46-484b-a00a-51e644fceacc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57d91c68cb8542b598108e6bf4152800', 'request_id': 'req-15cdeafd-ce46-484b-a00a-51e644fceacc'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:15.088319 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-15cdeafd-ce46-484b-a00a-51e644fceacc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57d91c68cb8542b598108e6bf4152800', 'request_id': 'req-15cdeafd-ce46-484b-a00a-51e644fceacc'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:15.156130 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-15cdeafd-ce46-484b-a00a-51e644fceacc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57d91c68cb8542b598108e6bf4152800', 'request_id': 'req-15cdeafd-ce46-484b-a00a-51e644fceacc'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:15.168196 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-15cdeafd-ce46-484b-a00a-51e644fceacc admin admin] The token cache is being invalidate because project 57d91c68cb8542b598108e6bf4152800 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:51:15.168582 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-15cdeafd-ce46-484b-a00a-51e644fceacc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:15.172730 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-15cdeafd-ce46-484b-a00a-51e644fceacc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:15.172730 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1428/2861] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:51:14 2022] DELETE /identity/v3/projects/57d91c68cb8542b598108e6bf4152800 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:51:15.174834 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fee8d0a-6227-442f-bf0e-752850ca8869 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:51:15.176253 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-7fee8d0a-6227-442f-bf0e-752850ca8869 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:51:15.189490 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fee8d0a-6227-442f-bf0e-752850ca8869 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:51:15.244286 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996644e1-f799-41f1-96bb-f87677a330f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:51:15.244951 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996644e1-f799-41f1-96bb-f87677a330f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:15.245183 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996644e1-f799-41f1-96bb-f87677a330f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:15.245459 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996644e1-f799-41f1-96bb-f87677a330f1 admin admin] PATH_INFO: `/v3/users/2e404445f35a4d23a5dda5d171a66e9f` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:15.247112 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996644e1-f799-41f1-96bb-f87677a330f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e404445f35a4d23a5dda5d171a66e9f)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:51:15.249023 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996644e1-f799-41f1-96bb-f87677a330f1 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:51:15.260899 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:51:15.455473 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-996644e1-f799-41f1-96bb-f87677a330f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e404445f35a4d23a5dda5d171a66e9f'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:15.528011 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-996644e1-f799-41f1-96bb-f87677a330f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e404445f35a4d23a5dda5d171a66e9f'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:15.528643 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-996644e1-f799-41f1-96bb-f87677a330f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e404445f35a4d23a5dda5d171a66e9f'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:15.529253 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-996644e1-f799-41f1-96bb-f87677a330f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e404445f35a4d23a5dda5d171a66e9f'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:15.536785 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-996644e1-f799-41f1-96bb-f87677a330f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e404445f35a4d23a5dda5d171a66e9f'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:15.554551 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1434/2862] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:51:15 2022] DELETE /identity/v3/users/2e404445f35a4d23a5dda5d171a66e9f => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:51:15.997417 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cdeafd-ce46-484b-a00a-51e644fceacc admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:51:15.998915 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-15cdeafd-ce46-484b-a00a-51e644fceacc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:51:16.014022 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cdeafd-ce46-484b-a00a-51e644fceacc admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:51:16.072568 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850b4315-8029-4f54-a3dc-3bfa0249609d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:51:16.073204 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850b4315-8029-4f54-a3dc-3bfa0249609d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:16.073488 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850b4315-8029-4f54-a3dc-3bfa0249609d admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:16.073812 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850b4315-8029-4f54-a3dc-3bfa0249609d admin admin] PATH_INFO: `/v3/projects/1a8aa7d71de44cb7b2b1821fbd4f3c77` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:16.075652 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850b4315-8029-4f54-a3dc-3bfa0249609d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a8aa7d71de44cb7b2b1821fbd4f3c77)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:51:16.077557 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850b4315-8029-4f54-a3dc-3bfa0249609d admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:51:16.189534 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:51:16.190940 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-850b4315-8029-4f54-a3dc-3bfa0249609d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a8aa7d71de44cb7b2b1821fbd4f3c77', 'request_id': 'req-850b4315-8029-4f54-a3dc-3bfa0249609d'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:16.197048 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-850b4315-8029-4f54-a3dc-3bfa0249609d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a8aa7d71de44cb7b2b1821fbd4f3c77', 'request_id': 'req-850b4315-8029-4f54-a3dc-3bfa0249609d'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:16.264571 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-850b4315-8029-4f54-a3dc-3bfa0249609d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a8aa7d71de44cb7b2b1821fbd4f3c77', 'request_id': 'req-850b4315-8029-4f54-a3dc-3bfa0249609d'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:16.275351 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-850b4315-8029-4f54-a3dc-3bfa0249609d admin admin] The token cache is being invalidate because project 1a8aa7d71de44cb7b2b1821fbd4f3c77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:51:16.280070 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-850b4315-8029-4f54-a3dc-3bfa0249609d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:16.280070 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-850b4315-8029-4f54-a3dc-3bfa0249609d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:16.280070 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1429/2863] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:51:15 2022] DELETE /identity/v3/projects/1a8aa7d71de44cb7b2b1821fbd4f3c77 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:51:16.284664 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69150c96-bff1-44ce-83eb-20b1ff8df062 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:16.285126 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69150c96-bff1-44ce-83eb-20b1ff8df062 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:16.285231 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69150c96-bff1-44ce-83eb-20b1ff8df062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:16.305786 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-69150c96-bff1-44ce-83eb-20b1ff8df062 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:51:16.306994 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-69150c96-bff1-44ce-83eb-20b1ff8df062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:51:16.365411 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1435/2864] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:51:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:51:16.375127 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2e2056-db75-4268-b19b-808d77b9ab61 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:16.375533 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2e2056-db75-4268-b19b-808d77b9ab61 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:16.375817 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2e2056-db75-4268-b19b-808d77b9ab61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:16.395683 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-ee2e2056-db75-4268-b19b-808d77b9ab61 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:51:16.396866 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-ee2e2056-db75-4268-b19b-808d77b9ab61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:51:16.405166 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1430/2865] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:51:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:51:16.409291 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69150c96-bff1-44ce-83eb-20b1ff8df062 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:51:16.419095 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69150c96-bff1-44ce-83eb-20b1ff8df062 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:51:16.423655 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec4f529-73dc-4768-9b18-1a4312c3426c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:51:16.424252 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec4f529-73dc-4768-9b18-1a4312c3426c admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:16.424535 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec4f529-73dc-4768-9b18-1a4312c3426c admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:16.424811 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec4f529-73dc-4768-9b18-1a4312c3426c admin admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:16.425474 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec4f529-73dc-4768-9b18-1a4312c3426c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:51:16.425874 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec4f529-73dc-4768-9b18-1a4312c3426c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:51:16.427465 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec4f529-73dc-4768-9b18-1a4312c3426c admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:51:16.439541 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1436/2866] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 15 07:51:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 07:51:16.443745 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2e2056-db75-4268-b19b-808d77b9ab61 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:51:16.453431 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2e2056-db75-4268-b19b-808d77b9ab61 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:51:16.458113 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825d9cdb-8456-48ea-a876-0371dfd56b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:51:16.458965 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825d9cdb-8456-48ea-a876-0371dfd56b5d admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:16.459247 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825d9cdb-8456-48ea-a876-0371dfd56b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:16.459521 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825d9cdb-8456-48ea-a876-0371dfd56b5d admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:16.460158 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825d9cdb-8456-48ea-a876-0371dfd56b5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:51:16.462086 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825d9cdb-8456-48ea-a876-0371dfd56b5d admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:51:16.586102 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1431/2867] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:51:16 2022] POST /identity/v3/projects => generated 383 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 07:51:16.590561 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec4f529-73dc-4768-9b18-1a4312c3426c admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:51:16.600686 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec4f529-73dc-4768-9b18-1a4312c3426c admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:51:16.658021 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffb38b8-d29e-4e42-91cc-ee81eef8e163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:51:16.658637 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffb38b8-d29e-4e42-91cc-ee81eef8e163 admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:16.658993 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffb38b8-d29e-4e42-91cc-ee81eef8e163 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:16.659281 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffb38b8-d29e-4e42-91cc-ee81eef8e163 admin admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:16.659993 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffb38b8-d29e-4e42-91cc-ee81eef8e163 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:51:16.662043 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffb38b8-d29e-4e42-91cc-ee81eef8e163 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:51:16.721956 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1437/2868] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:51:16 2022] POST /identity/v3/users => generated 292 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:51:16.726606 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825d9cdb-8456-48ea-a876-0371dfd56b5d admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:51:16.737730 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825d9cdb-8456-48ea-a876-0371dfd56b5d admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:51:16.742505 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b47245-1acf-4dc1-9b2b-d6c2a39cd417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:51:16.743141 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b47245-1acf-4dc1-9b2b-d6c2a39cd417 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:16.743450 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b47245-1acf-4dc1-9b2b-d6c2a39cd417 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:16.743729 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b47245-1acf-4dc1-9b2b-d6c2a39cd417 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:16.744378 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b47245-1acf-4dc1-9b2b-d6c2a39cd417 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:51:16.746628 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b47245-1acf-4dc1-9b2b-d6c2a39cd417 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:51:16.756676 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1432/2869] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:51:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:51:16.760654 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffb38b8-d29e-4e42-91cc-ee81eef8e163 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:51:16.771317 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffb38b8-d29e-4e42-91cc-ee81eef8e163 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:51:16.775989 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa78e561-e093-4c6d-a05e-2e61ca8d3483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:51:16.776582 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa78e561-e093-4c6d-a05e-2e61ca8d3483 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:16.776864 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa78e561-e093-4c6d-a05e-2e61ca8d3483 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:16.777211 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa78e561-e093-4c6d-a05e-2e61ca8d3483 admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:16.778015 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa78e561-e093-4c6d-a05e-2e61ca8d3483 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:51:16.779592 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa78e561-e093-4c6d-a05e-2e61ca8d3483 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:51:16.780846 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:51:16.790667 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1438/2870] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:51:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:51:16.795107 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b47245-1acf-4dc1-9b2b-d6c2a39cd417 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:51:16.805529 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b47245-1acf-4dc1-9b2b-d6c2a39cd417 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:51:16.810782 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9599d022-6ab4-4653-a527-8ef9e505e8d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:51:16.815056 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9599d022-6ab4-4653-a527-8ef9e505e8d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:16.815056 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9599d022-6ab4-4653-a527-8ef9e505e8d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:16.815056 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9599d022-6ab4-4653-a527-8ef9e505e8d1 admin admin] PATH_INFO: `/v3/projects/f623c858ec4248d3a673901b0bb82fe5/users/0e1c3753105f40278587c140d534d3e7/roles/053e12af84e943e699460a9d76c2072b` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:16.828161 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9599d022-6ab4-4653-a527-8ef9e505e8d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f623c858ec4248d3a673901b0bb82fe5, user_id=0e1c3753105f40278587c140d534d3e7, role_id=053e12af84e943e699460a9d76c2072b)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:51:16.831116 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9599d022-6ab4-4653-a527-8ef9e505e8d1 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:51:16.909996 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1433/2871] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:51:16 2022] PUT /identity/v3/projects/f623c858ec4248d3a673901b0bb82fe5/users/0e1c3753105f40278587c140d534d3e7/roles/053e12af84e943e699460a9d76c2072b => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 07:51:16.921443 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade722df-7668-4e57-897e-54946e7f0fa5 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:16.921760 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade722df-7668-4e57-897e-54946e7f0fa5 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:16.922036 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade722df-7668-4e57-897e-54946e7f0fa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:16.950824 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-ade722df-7668-4e57-897e-54946e7f0fa5 None None] MFA Rules not processed for user `0e1c3753105f40278587c140d534d3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:51:16.952020 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-ade722df-7668-4e57-897e-54946e7f0fa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:51:17.015283 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1439/2872] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:51:16 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:51:17.026199 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9599d022-6ab4-4653-a527-8ef9e505e8d1 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:51:17.028147 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-9599d022-6ab4-4653-a527-8ef9e505e8d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:51:17.085011 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9599d022-6ab4-4653-a527-8ef9e505e8d1 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:51:17.088604 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8772c210-301a-430e-a5e9-6e77e1159b58 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecf6642b7b84d3c9756bb41ab3349d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:51:17.089201 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8772c210-301a-430e-a5e9-6e77e1159b58 service cinder] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:17.089476 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8772c210-301a-430e-a5e9-6e77e1159b58 service cinder] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:17.089804 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8772c210-301a-430e-a5e9-6e77e1159b58 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:17.098007 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8772c210-301a-430e-a5e9-6e77e1159b58 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:51:17.099409 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8772c210-301a-430e-a5e9-6e77e1159b58 service cinder] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:51:17.108593 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1434/2873] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:51:17 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:51:25.726308 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade722df-7668-4e57-897e-54946e7f0fa5 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:51:25.786901 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade722df-7668-4e57-897e-54946e7f0fa5 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:51:25.791347 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13422f2-e181-4b58-95a8-a2fa001ddccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:51:25.791973 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13422f2-e181-4b58-95a8-a2fa001ddccf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:25.792250 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13422f2-e181-4b58-95a8-a2fa001ddccf admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:25.792521 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13422f2-e181-4b58-95a8-a2fa001ddccf admin admin] PATH_INFO: `/v3/users/0e1c3753105f40278587c140d534d3e7` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:25.793635 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13422f2-e181-4b58-95a8-a2fa001ddccf admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e1c3753105f40278587c140d534d3e7)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:51:25.796434 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13422f2-e181-4b58-95a8-a2fa001ddccf admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:51:25.951733 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-c13422f2-e181-4b58-95a8-a2fa001ddccf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e1c3753105f40278587c140d534d3e7'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:26.013061 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-c13422f2-e181-4b58-95a8-a2fa001ddccf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e1c3753105f40278587c140d534d3e7'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:26.017088 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-c13422f2-e181-4b58-95a8-a2fa001ddccf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e1c3753105f40278587c140d534d3e7'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:26.017088 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-c13422f2-e181-4b58-95a8-a2fa001ddccf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e1c3753105f40278587c140d534d3e7'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:26.022246 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-c13422f2-e181-4b58-95a8-a2fa001ddccf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e1c3753105f40278587c140d534d3e7'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:26.039673 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1440/2874] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:51:25 2022] DELETE /identity/v3/users/0e1c3753105f40278587c140d534d3e7 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:51:26.049801 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cdc762-fae4-4d96-8a62-0f13569fc3b5 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:26.050248 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cdc762-fae4-4d96-8a62-0f13569fc3b5 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:26.050477 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cdc762-fae4-4d96-8a62-0f13569fc3b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:26.069849 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-69cdc762-fae4-4d96-8a62-0f13569fc3b5 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:51:26.071030 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-69cdc762-fae4-4d96-8a62-0f13569fc3b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:51:26.128365 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1435/2875] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:51:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:51:26.138160 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13422f2-e181-4b58-95a8-a2fa001ddccf admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:51:26.140095 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-c13422f2-e181-4b58-95a8-a2fa001ddccf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:51:26.146727 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:51:26.202327 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13422f2-e181-4b58-95a8-a2fa001ddccf admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:51:26.205966 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f81778-7fb7-4724-8a39-f9bf9fa6cccc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:51:26.206566 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f81778-7fb7-4724-8a39-f9bf9fa6cccc service neutron] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:26.206842 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f81778-7fb7-4724-8a39-f9bf9fa6cccc service neutron] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:26.207113 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f81778-7fb7-4724-8a39-f9bf9fa6cccc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:26.217478 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f81778-7fb7-4724-8a39-f9bf9fa6cccc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:51:26.219002 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f81778-7fb7-4724-8a39-f9bf9fa6cccc service neutron] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:51:26.228141 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1441/2876] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:51:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:51:26.597475 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cdc762-fae4-4d96-8a62-0f13569fc3b5 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:51:26.599010 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-69cdc762-fae4-4d96-8a62-0f13569fc3b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:51:26.611538 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cdc762-fae4-4d96-8a62-0f13569fc3b5 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:51:26.616466 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90252906-3d06-4b5d-ad1f-97a61e9a4f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:51:26.617105 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90252906-3d06-4b5d-ad1f-97a61e9a4f62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:26.617385 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90252906-3d06-4b5d-ad1f-97a61e9a4f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:26.617695 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90252906-3d06-4b5d-ad1f-97a61e9a4f62 admin admin] PATH_INFO: `/v3/projects/f623c858ec4248d3a673901b0bb82fe5` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:26.618805 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90252906-3d06-4b5d-ad1f-97a61e9a4f62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f623c858ec4248d3a673901b0bb82fe5)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:51:26.620691 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90252906-3d06-4b5d-ad1f-97a61e9a4f62 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:51:26.632778 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:51:26.760681 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-90252906-3d06-4b5d-ad1f-97a61e9a4f62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f623c858ec4248d3a673901b0bb82fe5', 'request_id': 'req-90252906-3d06-4b5d-ad1f-97a61e9a4f62'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:26.767782 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-90252906-3d06-4b5d-ad1f-97a61e9a4f62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f623c858ec4248d3a673901b0bb82fe5', 'request_id': 'req-90252906-3d06-4b5d-ad1f-97a61e9a4f62'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:26.835522 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-90252906-3d06-4b5d-ad1f-97a61e9a4f62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f623c858ec4248d3a673901b0bb82fe5', 'request_id': 'req-90252906-3d06-4b5d-ad1f-97a61e9a4f62'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:26.847005 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-90252906-3d06-4b5d-ad1f-97a61e9a4f62 admin admin] The token cache is being invalidate because project f623c858ec4248d3a673901b0bb82fe5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:51:26.847245 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-90252906-3d06-4b5d-ad1f-97a61e9a4f62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:26.847883 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-90252906-3d06-4b5d-ad1f-97a61e9a4f62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:26.849349 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1436/2877] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:51:26 2022] DELETE /identity/v3/projects/f623c858ec4248d3a673901b0bb82fe5 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:51:26.854145 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d039cc58-9a92-44e3-a88c-a04d1e6d0061 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:26.854607 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d039cc58-9a92-44e3-a88c-a04d1e6d0061 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:26.854725 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d039cc58-9a92-44e3-a88c-a04d1e6d0061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:26.874335 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-d039cc58-9a92-44e3-a88c-a04d1e6d0061 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:51:26.875504 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-d039cc58-9a92-44e3-a88c-a04d1e6d0061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:51:26.933371 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1442/2878] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:51:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:51:26.941974 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12326efb-a4db-4b7a-9e1b-274bd368b53c None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:26.942264 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12326efb-a4db-4b7a-9e1b-274bd368b53c None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:26.942840 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12326efb-a4db-4b7a-9e1b-274bd368b53c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:26.963561 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-12326efb-a4db-4b7a-9e1b-274bd368b53c None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:51:26.964757 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-12326efb-a4db-4b7a-9e1b-274bd368b53c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:51:26.972707 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1437/2879] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:51:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:51:26.976878 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d039cc58-9a92-44e3-a88c-a04d1e6d0061 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:51:26.986509 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d039cc58-9a92-44e3-a88c-a04d1e6d0061 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:51:26.990982 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef7dcc3-c888-45d4-bb31-166202a5fd15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:51:26.991580 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef7dcc3-c888-45d4-bb31-166202a5fd15 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:26.991857 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef7dcc3-c888-45d4-bb31-166202a5fd15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:26.992126 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef7dcc3-c888-45d4-bb31-166202a5fd15 admin admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:26.992760 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef7dcc3-c888-45d4-bb31-166202a5fd15 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:51:26.993061 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef7dcc3-c888-45d4-bb31-166202a5fd15 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:51:26.994687 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef7dcc3-c888-45d4-bb31-166202a5fd15 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:51:27.006469 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1443/2880] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 15 07:51:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:51:27.010644 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12326efb-a4db-4b7a-9e1b-274bd368b53c None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:51:27.020453 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12326efb-a4db-4b7a-9e1b-274bd368b53c None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:51:27.025052 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7f3077-9e5b-438e-8af8-16ebc7261ff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:51:27.025678 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7f3077-9e5b-438e-8af8-16ebc7261ff1 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:27.025963 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7f3077-9e5b-438e-8af8-16ebc7261ff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:27.026239 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7f3077-9e5b-438e-8af8-16ebc7261ff1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:27.026879 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7f3077-9e5b-438e-8af8-16ebc7261ff1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:51:27.028757 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7f3077-9e5b-438e-8af8-16ebc7261ff1 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:51:27.110810 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1438/2881] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:51:27 2022] POST /identity/v3/projects => generated 375 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:51:27.114957 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef7dcc3-c888-45d4-bb31-166202a5fd15 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:51:27.125474 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef7dcc3-c888-45d4-bb31-166202a5fd15 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:51:27.180523 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4309658b-0960-408b-8745-235cadaa8712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:51:27.181127 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4309658b-0960-408b-8745-235cadaa8712 admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:27.181406 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4309658b-0960-408b-8745-235cadaa8712 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:27.181714 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4309658b-0960-408b-8745-235cadaa8712 admin admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:27.182345 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4309658b-0960-408b-8745-235cadaa8712 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:51:27.184250 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4309658b-0960-408b-8745-235cadaa8712 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:51:27.285992 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1444/2882] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:51:27 2022] POST /identity/v3/users => generated 288 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:51:27.291522 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7f3077-9e5b-438e-8af8-16ebc7261ff1 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:51:27.302735 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7f3077-9e5b-438e-8af8-16ebc7261ff1 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:51:27.307435 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fa6952-9236-498c-b946-86cecab04a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:51:27.308028 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fa6952-9236-498c-b946-86cecab04a17 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:27.308415 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fa6952-9236-498c-b946-86cecab04a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:27.308698 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fa6952-9236-498c-b946-86cecab04a17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:27.309341 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fa6952-9236-498c-b946-86cecab04a17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:51:27.311580 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fa6952-9236-498c-b946-86cecab04a17 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:51:27.321484 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1439/2883] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:51:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:51:27.325305 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4309658b-0960-408b-8745-235cadaa8712 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:51:27.335687 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4309658b-0960-408b-8745-235cadaa8712 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:51:27.340237 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c0fea1-69d9-433f-bf7e-d4b845aaeb8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:51:27.340824 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c0fea1-69d9-433f-bf7e-d4b845aaeb8d admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:27.341101 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c0fea1-69d9-433f-bf7e-d4b845aaeb8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:27.341371 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c0fea1-69d9-433f-bf7e-d4b845aaeb8d admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:27.342070 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c0fea1-69d9-433f-bf7e-d4b845aaeb8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:51:27.343641 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c0fea1-69d9-433f-bf7e-d4b845aaeb8d admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:51:27.355061 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1445/2884] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:51:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:51:27.359724 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fa6952-9236-498c-b946-86cecab04a17 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:51:27.370086 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fa6952-9236-498c-b946-86cecab04a17 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:51:27.375447 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9526de10-0490-4e19-89af-0d6f98b72941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:51:27.376098 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9526de10-0490-4e19-89af-0d6f98b72941 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:27.376378 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9526de10-0490-4e19-89af-0d6f98b72941 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:27.376654 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9526de10-0490-4e19-89af-0d6f98b72941 admin admin] PATH_INFO: `/v3/projects/18880a07f19645369d9d0937f6256538/users/29c817505c5d436c87267692b9867b81/roles/053e12af84e943e699460a9d76c2072b` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:27.391490 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9526de10-0490-4e19-89af-0d6f98b72941 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18880a07f19645369d9d0937f6256538, user_id=29c817505c5d436c87267692b9867b81, role_id=053e12af84e943e699460a9d76c2072b)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:51:27.394240 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9526de10-0490-4e19-89af-0d6f98b72941 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:51:27.459167 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1440/2885] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:51:27 2022] PUT /identity/v3/projects/18880a07f19645369d9d0937f6256538/users/29c817505c5d436c87267692b9867b81/roles/053e12af84e943e699460a9d76c2072b => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:51:27.470036 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ad8082-1693-43b0-b744-4b91e1bc338e None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:27.470345 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ad8082-1693-43b0-b744-4b91e1bc338e None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:27.470613 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ad8082-1693-43b0-b744-4b91e1bc338e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:27.495849 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:51:27.499742 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-63ad8082-1693-43b0-b744-4b91e1bc338e None None] MFA Rules not processed for user `29c817505c5d436c87267692b9867b81`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:51:27.500983 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-63ad8082-1693-43b0-b744-4b91e1bc338e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:51:27.565159 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1446/2886] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:51:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:51:27.569807 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9526de10-0490-4e19-89af-0d6f98b72941 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:51:27.631274 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9526de10-0490-4e19-89af-0d6f98b72941 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:51:27.636045 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb5d61b-761e-4009-89bb-46cc00df5b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:51:27.636654 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb5d61b-761e-4009-89bb-46cc00df5b56 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:27.636934 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb5d61b-761e-4009-89bb-46cc00df5b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:27.637209 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb5d61b-761e-4009-89bb-46cc00df5b56 admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:27.637888 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb5d61b-761e-4009-89bb-46cc00df5b56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:51:27.640229 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb5d61b-761e-4009-89bb-46cc00df5b56 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:51:27.728348 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1441/2887] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:51:27 2022] POST /identity/v3/projects => generated 377 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:51:27.734380 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ad8082-1693-43b0-b744-4b91e1bc338e None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:51:27.743965 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ad8082-1693-43b0-b744-4b91e1bc338e None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:51:27.799423 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f37bce-3fc8-487a-a968-a2529451b7fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:51:27.800027 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f37bce-3fc8-487a-a968-a2529451b7fc admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:27.800306 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f37bce-3fc8-487a-a968-a2529451b7fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:27.800575 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f37bce-3fc8-487a-a968-a2529451b7fc admin admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:27.801193 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f37bce-3fc8-487a-a968-a2529451b7fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:51:27.803132 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f37bce-3fc8-487a-a968-a2529451b7fc admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:51:27.892909 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1447/2888] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:51:27 2022] POST /identity/v3/users => generated 289 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:51:27.896969 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb5d61b-761e-4009-89bb-46cc00df5b56 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:51:27.910207 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb5d61b-761e-4009-89bb-46cc00df5b56 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:51:27.914941 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28cf127-9e45-46ae-886f-262bb7f08adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:51:27.915547 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28cf127-9e45-46ae-886f-262bb7f08adf admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:27.915834 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28cf127-9e45-46ae-886f-262bb7f08adf admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:27.916112 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28cf127-9e45-46ae-886f-262bb7f08adf admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:27.916764 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28cf127-9e45-46ae-886f-262bb7f08adf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:51:27.918385 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28cf127-9e45-46ae-886f-262bb7f08adf admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:51:27.929819 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1442/2889] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:51:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:51:27.933844 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f37bce-3fc8-487a-a968-a2529451b7fc admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:51:27.944289 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f37bce-3fc8-487a-a968-a2529451b7fc admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:51:27.948891 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d194a695-2281-400c-a4ec-51e7644f9483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:51:27.949492 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d194a695-2281-400c-a4ec-51e7644f9483 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:27.949793 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d194a695-2281-400c-a4ec-51e7644f9483 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:27.950065 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d194a695-2281-400c-a4ec-51e7644f9483 admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:27.950699 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d194a695-2281-400c-a4ec-51e7644f9483 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:51:27.952510 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d194a695-2281-400c-a4ec-51e7644f9483 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:51:27.962836 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1448/2890] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:51:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 07:51:27.967605 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28cf127-9e45-46ae-886f-262bb7f08adf admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:51:27.977714 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28cf127-9e45-46ae-886f-262bb7f08adf admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:51:27.983293 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ef6267-17ac-45bb-9d73-3506c3004d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:51:27.984073 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ef6267-17ac-45bb-9d73-3506c3004d47 admin admin] REQUEST_METHOD: `PUT` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:27.984359 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ef6267-17ac-45bb-9d73-3506c3004d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:27.984639 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ef6267-17ac-45bb-9d73-3506c3004d47 admin admin] PATH_INFO: `/v3/projects/a9136e2f236f4fd5b5512dc748fc4fb2/users/11f888d0b2f64552ac922e93f71d4d5a/roles/053e12af84e943e699460a9d76c2072b` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:27.987718 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:51:27.999269 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ef6267-17ac-45bb-9d73-3506c3004d47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9136e2f236f4fd5b5512dc748fc4fb2, user_id=11f888d0b2f64552ac922e93f71d4d5a, role_id=053e12af84e943e699460a9d76c2072b)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:51:28.001966 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ef6267-17ac-45bb-9d73-3506c3004d47 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:51:28.058487 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1443/2891] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:51:27 2022] PUT /identity/v3/projects/a9136e2f236f4fd5b5512dc748fc4fb2/users/11f888d0b2f64552ac922e93f71d4d5a/roles/053e12af84e943e699460a9d76c2072b => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:51:28.068990 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60700b7f-df9d-4df7-aeb6-e97fb9aba909 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:28.069278 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60700b7f-df9d-4df7-aeb6-e97fb9aba909 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:28.069556 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60700b7f-df9d-4df7-aeb6-e97fb9aba909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:28.098665 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-60700b7f-df9d-4df7-aeb6-e97fb9aba909 None None] MFA Rules not processed for user `11f888d0b2f64552ac922e93f71d4d5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:51:28.099853 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-60700b7f-df9d-4df7-aeb6-e97fb9aba909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:51:28.164225 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1449/2892] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:51:28 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:51:28.168705 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ef6267-17ac-45bb-9d73-3506c3004d47 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:51:28.245384 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ef6267-17ac-45bb-9d73-3506c3004d47 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:51:28.251164 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9fbf46-9c8a-4c18-9036-c291a0adddd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:51:28.251829 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9fbf46-9c8a-4c18-9036-c291a0adddd2 admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:28.252113 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9fbf46-9c8a-4c18-9036-c291a0adddd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:28.252385 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9fbf46-9c8a-4c18-9036-c291a0adddd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:28.253045 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9fbf46-9c8a-4c18-9036-c291a0adddd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:51:28.254976 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9fbf46-9c8a-4c18-9036-c291a0adddd2 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:51:28.328881 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1444/2893] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:51:28 2022] POST /identity/v3/projects => generated 377 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:51:28.333548 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60700b7f-df9d-4df7-aeb6-e97fb9aba909 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:51:28.345084 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60700b7f-df9d-4df7-aeb6-e97fb9aba909 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:51:28.352765 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:51:28.400858 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a8562d-1eb6-4750-b7b9-33708e3babe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:51:28.401464 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a8562d-1eb6-4750-b7b9-33708e3babe9 admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:28.401772 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a8562d-1eb6-4750-b7b9-33708e3babe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:28.402042 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a8562d-1eb6-4750-b7b9-33708e3babe9 admin admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:28.402659 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a8562d-1eb6-4750-b7b9-33708e3babe9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:51:28.404604 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a8562d-1eb6-4750-b7b9-33708e3babe9 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:51:28.498815 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1450/2894] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:51:28 2022] POST /identity/v3/users => generated 295 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:51:28.502880 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9fbf46-9c8a-4c18-9036-c291a0adddd2 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:51:28.513379 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9fbf46-9c8a-4c18-9036-c291a0adddd2 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:51:28.518539 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae29f73f-ee6d-440a-bdb4-61d14c04fa0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:51:28.519138 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae29f73f-ee6d-440a-bdb4-61d14c04fa0e admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:28.519458 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae29f73f-ee6d-440a-bdb4-61d14c04fa0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:28.519727 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae29f73f-ee6d-440a-bdb4-61d14c04fa0e admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:28.520402 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae29f73f-ee6d-440a-bdb4-61d14c04fa0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:51:28.522057 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae29f73f-ee6d-440a-bdb4-61d14c04fa0e admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:51:28.533654 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1445/2895] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:51:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:51:28.537732 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a8562d-1eb6-4750-b7b9-33708e3babe9 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:51:28.547736 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a8562d-1eb6-4750-b7b9-33708e3babe9 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:51:28.559407 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f3ff1e-8737-4cd0-aa6e-df574300c0ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:51:28.560049 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f3ff1e-8737-4cd0-aa6e-df574300c0ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:28.560324 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f3ff1e-8737-4cd0-aa6e-df574300c0ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:28.561103 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f3ff1e-8737-4cd0-aa6e-df574300c0ba admin admin] PATH_INFO: `/v3/projects/fc7cf240f39246578637e6b799607f29/users/970c77dbcd2c4c048d5c28f95dd22808/roles/6bbb643e0d8d4bc0a9b27c26edf59216` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:28.580980 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f3ff1e-8737-4cd0-aa6e-df574300c0ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc7cf240f39246578637e6b799607f29, user_id=970c77dbcd2c4c048d5c28f95dd22808, role_id=6bbb643e0d8d4bc0a9b27c26edf59216)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:51:28.583933 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f3ff1e-8737-4cd0-aa6e-df574300c0ba admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:51:28.651978 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1451/2896] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:51:28 2022] PUT /identity/v3/projects/fc7cf240f39246578637e6b799607f29/users/970c77dbcd2c4c048d5c28f95dd22808/roles/6bbb643e0d8d4bc0a9b27c26edf59216 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:51:28.656249 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae29f73f-ee6d-440a-bdb4-61d14c04fa0e admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:51:28.666437 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae29f73f-ee6d-440a-bdb4-61d14c04fa0e admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:51:28.724561 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a1728d-797c-47a4-b4de-0c914b99ebe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:51:28.725167 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a1728d-797c-47a4-b4de-0c914b99ebe2 admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:28.725445 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a1728d-797c-47a4-b4de-0c914b99ebe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:28.725754 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a1728d-797c-47a4-b4de-0c914b99ebe2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:28.726393 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a1728d-797c-47a4-b4de-0c914b99ebe2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:51:28.727962 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a1728d-797c-47a4-b4de-0c914b99ebe2 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:51:28.738379 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1446/2897] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:51:28 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 07:51:28.742509 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f3ff1e-8737-4cd0-aa6e-df574300c0ba admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:51:28.758686 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f3ff1e-8737-4cd0-aa6e-df574300c0ba admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:51:28.758686 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d3c1de-ec9c-4947-9882-d97a9c21fc95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:51:28.758686 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d3c1de-ec9c-4947-9882-d97a9c21fc95 admin admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:28.758686 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d3c1de-ec9c-4947-9882-d97a9c21fc95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:28.759146 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d3c1de-ec9c-4947-9882-d97a9c21fc95 admin admin] PATH_INFO: `/v3/projects/fc7cf240f39246578637e6b799607f29/users/970c77dbcd2c4c048d5c28f95dd22808/roles/6bbb643e0d8d4bc0a9b27c26edf59216` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:28.760746 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d3c1de-ec9c-4947-9882-d97a9c21fc95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc7cf240f39246578637e6b799607f29, user_id=970c77dbcd2c4c048d5c28f95dd22808, role_id=6bbb643e0d8d4bc0a9b27c26edf59216)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:51:28.763423 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d3c1de-ec9c-4947-9882-d97a9c21fc95 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:51:28.774117 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1452/2898] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:51:28 2022] PUT /identity/v3/projects/fc7cf240f39246578637e6b799607f29/users/970c77dbcd2c4c048d5c28f95dd22808/roles/6bbb643e0d8d4bc0a9b27c26edf59216 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:51:28.786029 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6ccdd4-81dc-4b4c-9324-c22239003356 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:28.786314 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6ccdd4-81dc-4b4c-9324-c22239003356 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:28.786612 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6ccdd4-81dc-4b4c-9324-c22239003356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:28.814250 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-7a6ccdd4-81dc-4b4c-9324-c22239003356 None None] MFA Rules not processed for user `970c77dbcd2c4c048d5c28f95dd22808`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:51:28.816465 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-7a6ccdd4-81dc-4b4c-9324-c22239003356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:51:28.861731 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:51:28.892485 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1447/2899] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:51:28 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 07:51:28.904601 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d3c1de-ec9c-4947-9882-d97a9c21fc95 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:51:28.906469 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-b1d3c1de-ec9c-4947-9882-d97a9c21fc95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:51:28.964907 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d3c1de-ec9c-4947-9882-d97a9c21fc95 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:51:28.969338 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79530872-c932-443d-a3f1-62d9b91c27a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecf6642b7b84d3c9756bb41ab3349d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:51:28.969973 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79530872-c932-443d-a3f1-62d9b91c27a1 service cinder] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:28.970252 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79530872-c932-443d-a3f1-62d9b91c27a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:28.970522 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79530872-c932-443d-a3f1-62d9b91c27a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:29.025415 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79530872-c932-443d-a3f1-62d9b91c27a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:51:29.026791 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79530872-c932-443d-a3f1-62d9b91c27a1 service cinder] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:51:29.036243 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1453/2900] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:51:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:51:30.991931 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6ccdd4-81dc-4b4c-9324-c22239003356 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:51:30.993331 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-7a6ccdd4-81dc-4b4c-9324-c22239003356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:51:31.006263 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6ccdd4-81dc-4b4c-9324-c22239003356 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:51:31.009664 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15473dc-e017-4c2f-9bc2-1e194692e2f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecf6642b7b84d3c9756bb41ab3349d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:51:31.010270 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15473dc-e017-4c2f-9bc2-1e194692e2f9 service cinder] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:31.010550 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15473dc-e017-4c2f-9bc2-1e194692e2f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:31.010823 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15473dc-e017-4c2f-9bc2-1e194692e2f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:31.066590 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15473dc-e017-4c2f-9bc2-1e194692e2f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:51:31.067998 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15473dc-e017-4c2f-9bc2-1e194692e2f9 service cinder] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:51:31.076863 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1448/2901] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:51:30 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:51:32.240356 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79530872-c932-443d-a3f1-62d9b91c27a1 service cinder] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:51:32.249695 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79530872-c932-443d-a3f1-62d9b91c27a1 service cinder] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:51:32.254223 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5dece6-7230-430a-a4f5-ff204bb6da6a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecf6642b7b84d3c9756bb41ab3349d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:51:32.254223 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5dece6-7230-430a-a4f5-ff204bb6da6a service cinder] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:32.254223 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5dece6-7230-430a-a4f5-ff204bb6da6a service cinder] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:32.254562 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5dece6-7230-430a-a4f5-ff204bb6da6a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:32.263145 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5dece6-7230-430a-a4f5-ff204bb6da6a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:51:32.265150 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5dece6-7230-430a-a4f5-ff204bb6da6a service cinder] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:51:32.274673 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1454/2902] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:51:32 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:51:36.761476 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15473dc-e017-4c2f-9bc2-1e194692e2f9 service cinder] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:51:36.825125 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15473dc-e017-4c2f-9bc2-1e194692e2f9 service cinder] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:51:36.829859 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98617d9f-b1b9-4147-9837-e8276baf1941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:51:36.830490 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98617d9f-b1b9-4147-9837-e8276baf1941 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:36.830777 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98617d9f-b1b9-4147-9837-e8276baf1941 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:36.838343 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98617d9f-b1b9-4147-9837-e8276baf1941 admin admin] PATH_INFO: `/v3/users/29c817505c5d436c87267692b9867b81` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:36.838343 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98617d9f-b1b9-4147-9837-e8276baf1941 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29c817505c5d436c87267692b9867b81)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:51:36.838343 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98617d9f-b1b9-4147-9837-e8276baf1941 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:51:37.003867 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-98617d9f-b1b9-4147-9837-e8276baf1941 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29c817505c5d436c87267692b9867b81'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:37.004524 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-98617d9f-b1b9-4147-9837-e8276baf1941 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29c817505c5d436c87267692b9867b81'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:37.092035 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-98617d9f-b1b9-4147-9837-e8276baf1941 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29c817505c5d436c87267692b9867b81'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:37.093562 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-98617d9f-b1b9-4147-9837-e8276baf1941 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29c817505c5d436c87267692b9867b81'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:37.101405 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-98617d9f-b1b9-4147-9837-e8276baf1941 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29c817505c5d436c87267692b9867b81'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:37.114169 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:51:37.119115 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1449/2903] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:51:36 2022] DELETE /identity/v3/users/29c817505c5d436c87267692b9867b81 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:51:37.123661 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087bc16a-ec58-4315-9c89-1d3dc69961bd None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:37.124277 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087bc16a-ec58-4315-9c89-1d3dc69961bd None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:37.124393 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087bc16a-ec58-4315-9c89-1d3dc69961bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:37.132557 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:51:37.143805 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-087bc16a-ec58-4315-9c89-1d3dc69961bd None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:51:37.145001 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-087bc16a-ec58-4315-9c89-1d3dc69961bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:51:37.203609 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1455/2904] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:51:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:51:37.213597 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98617d9f-b1b9-4147-9837-e8276baf1941 admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:51:37.215045 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-98617d9f-b1b9-4147-9837-e8276baf1941 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:51:37.274110 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98617d9f-b1b9-4147-9837-e8276baf1941 admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:51:37.277740 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae34ac2-92f4-428a-8943-e4d48a7b7285 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:51:37.278345 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae34ac2-92f4-428a-8943-e4d48a7b7285 service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:37.278628 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae34ac2-92f4-428a-8943-e4d48a7b7285 service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:37.278907 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae34ac2-92f4-428a-8943-e4d48a7b7285 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:37.288760 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae34ac2-92f4-428a-8943-e4d48a7b7285 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:51:37.290216 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae34ac2-92f4-428a-8943-e4d48a7b7285 service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:51:37.299993 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1450/2905] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:51:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:51:37.677547 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087bc16a-ec58-4315-9c89-1d3dc69961bd None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:51:37.678995 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-087bc16a-ec58-4315-9c89-1d3dc69961bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:51:37.696102 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087bc16a-ec58-4315-9c89-1d3dc69961bd None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:51:37.697850 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a095571d-044f-420d-aa75-e5577e5cd6ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:51:37.698566 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a095571d-044f-420d-aa75-e5577e5cd6ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:37.698850 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a095571d-044f-420d-aa75-e5577e5cd6ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:37.699145 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a095571d-044f-420d-aa75-e5577e5cd6ec admin admin] PATH_INFO: `/v3/projects/18880a07f19645369d9d0937f6256538` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:37.700357 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a095571d-044f-420d-aa75-e5577e5cd6ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=18880a07f19645369d9d0937f6256538)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:51:37.702560 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a095571d-044f-420d-aa75-e5577e5cd6ec admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:51:37.831225 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-a095571d-044f-420d-aa75-e5577e5cd6ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18880a07f19645369d9d0937f6256538', 'request_id': 'req-a095571d-044f-420d-aa75-e5577e5cd6ec'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:37.837803 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-a095571d-044f-420d-aa75-e5577e5cd6ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18880a07f19645369d9d0937f6256538', 'request_id': 'req-a095571d-044f-420d-aa75-e5577e5cd6ec'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:37.903839 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-a095571d-044f-420d-aa75-e5577e5cd6ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18880a07f19645369d9d0937f6256538', 'request_id': 'req-a095571d-044f-420d-aa75-e5577e5cd6ec'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:37.915744 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-a095571d-044f-420d-aa75-e5577e5cd6ec admin admin] The token cache is being invalidate because project 18880a07f19645369d9d0937f6256538 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:51:37.916076 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-a095571d-044f-420d-aa75-e5577e5cd6ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:37.916730 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-a095571d-044f-420d-aa75-e5577e5cd6ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:37.918251 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1456/2906] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:51:37 2022] DELETE /identity/v3/projects/18880a07f19645369d9d0937f6256538 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:51:37.922512 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae34ac2-92f4-428a-8943-e4d48a7b7285 service neutron] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:51:37.932169 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae34ac2-92f4-428a-8943-e4d48a7b7285 service neutron] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:51:37.989612 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572c40f4-4943-4e9f-93fa-9b72a85d976f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:51:37.990267 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572c40f4-4943-4e9f-93fa-9b72a85d976f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:37.990553 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572c40f4-4943-4e9f-93fa-9b72a85d976f admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:37.990828 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572c40f4-4943-4e9f-93fa-9b72a85d976f admin admin] PATH_INFO: `/v3/users/11f888d0b2f64552ac922e93f71d4d5a` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:37.991987 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572c40f4-4943-4e9f-93fa-9b72a85d976f admin admin] RBAC: Authorizing `identity:delete_user(user_id=11f888d0b2f64552ac922e93f71d4d5a)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:51:37.993945 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572c40f4-4943-4e9f-93fa-9b72a85d976f admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:51:38.163339 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-572c40f4-4943-4e9f-93fa-9b72a85d976f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11f888d0b2f64552ac922e93f71d4d5a'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:38.163912 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-572c40f4-4943-4e9f-93fa-9b72a85d976f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11f888d0b2f64552ac922e93f71d4d5a'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:38.218615 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-572c40f4-4943-4e9f-93fa-9b72a85d976f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11f888d0b2f64552ac922e93f71d4d5a'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:38.219241 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-572c40f4-4943-4e9f-93fa-9b72a85d976f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11f888d0b2f64552ac922e93f71d4d5a'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:38.227201 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-572c40f4-4943-4e9f-93fa-9b72a85d976f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11f888d0b2f64552ac922e93f71d4d5a'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:38.246643 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1451/2907] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:51:37 2022] DELETE /identity/v3/users/11f888d0b2f64552ac922e93f71d4d5a => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:51:38.607275 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a095571d-044f-420d-aa75-e5577e5cd6ec admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:51:38.609873 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-a095571d-044f-420d-aa75-e5577e5cd6ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:51:38.623517 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a095571d-044f-420d-aa75-e5577e5cd6ec admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:51:38.682474 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f588d68a-493c-41a8-a513-42690c993e13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:51:38.683109 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f588d68a-493c-41a8-a513-42690c993e13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:38.683389 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f588d68a-493c-41a8-a513-42690c993e13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:38.683660 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f588d68a-493c-41a8-a513-42690c993e13 admin admin] PATH_INFO: `/v3/projects/a9136e2f236f4fd5b5512dc748fc4fb2` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:38.684981 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f588d68a-493c-41a8-a513-42690c993e13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9136e2f236f4fd5b5512dc748fc4fb2)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:51:38.686977 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f588d68a-493c-41a8-a513-42690c993e13 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:51:38.771557 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:51:38.784989 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-f588d68a-493c-41a8-a513-42690c993e13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9136e2f236f4fd5b5512dc748fc4fb2', 'request_id': 'req-f588d68a-493c-41a8-a513-42690c993e13'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:38.791146 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-f588d68a-493c-41a8-a513-42690c993e13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9136e2f236f4fd5b5512dc748fc4fb2', 'request_id': 'req-f588d68a-493c-41a8-a513-42690c993e13'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:38.857815 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-f588d68a-493c-41a8-a513-42690c993e13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9136e2f236f4fd5b5512dc748fc4fb2', 'request_id': 'req-f588d68a-493c-41a8-a513-42690c993e13'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:38.868395 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-f588d68a-493c-41a8-a513-42690c993e13 admin admin] The token cache is being invalidate because project a9136e2f236f4fd5b5512dc748fc4fb2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:51:38.868685 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-f588d68a-493c-41a8-a513-42690c993e13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:38.869313 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-f588d68a-493c-41a8-a513-42690c993e13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:38.870799 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1457/2908] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:51:38 2022] DELETE /identity/v3/projects/a9136e2f236f4fd5b5512dc748fc4fb2 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:51:38.875284 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572c40f4-4943-4e9f-93fa-9b72a85d976f admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:51:38.876709 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-572c40f4-4943-4e9f-93fa-9b72a85d976f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:51:38.889147 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572c40f4-4943-4e9f-93fa-9b72a85d976f admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:51:38.907266 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:51:38.946815 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013f6b45-d78e-42c3-8912-1d30ae6a1f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:51:38.947453 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f6b45-d78e-42c3-8912-1d30ae6a1f92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:38.947733 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f6b45-d78e-42c3-8912-1d30ae6a1f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:38.948076 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f6b45-d78e-42c3-8912-1d30ae6a1f92 admin admin] PATH_INFO: `/v3/users/970c77dbcd2c4c048d5c28f95dd22808` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:38.949283 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013f6b45-d78e-42c3-8912-1d30ae6a1f92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=970c77dbcd2c4c048d5c28f95dd22808)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:51:38.951240 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013f6b45-d78e-42c3-8912-1d30ae6a1f92 admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:51:39.127834 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-013f6b45-d78e-42c3-8912-1d30ae6a1f92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '970c77dbcd2c4c048d5c28f95dd22808'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:39.128506 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-013f6b45-d78e-42c3-8912-1d30ae6a1f92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '970c77dbcd2c4c048d5c28f95dd22808'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:39.188362 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-013f6b45-d78e-42c3-8912-1d30ae6a1f92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '970c77dbcd2c4c048d5c28f95dd22808'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:39.188988 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-013f6b45-d78e-42c3-8912-1d30ae6a1f92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '970c77dbcd2c4c048d5c28f95dd22808'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:39.196171 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-013f6b45-d78e-42c3-8912-1d30ae6a1f92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '970c77dbcd2c4c048d5c28f95dd22808'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:39.215255 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1452/2909] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:51:38 2022] DELETE /identity/v3/users/970c77dbcd2c4c048d5c28f95dd22808 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:51:39.614785 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f588d68a-493c-41a8-a513-42690c993e13 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:51:39.616315 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-f588d68a-493c-41a8-a513-42690c993e13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:51:39.629537 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f588d68a-493c-41a8-a513-42690c993e13 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:51:39.687697 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d96595d-e001-4d13-a180-b90f86320c5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:51:39.688328 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d96595d-e001-4d13-a180-b90f86320c5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:39.688604 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d96595d-e001-4d13-a180-b90f86320c5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:39.688877 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d96595d-e001-4d13-a180-b90f86320c5a admin admin] PATH_INFO: `/v3/projects/fc7cf240f39246578637e6b799607f29` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:39.690003 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d96595d-e001-4d13-a180-b90f86320c5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc7cf240f39246578637e6b799607f29)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:51:39.691974 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d96595d-e001-4d13-a180-b90f86320c5a admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:51:39.824066 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-7d96595d-e001-4d13-a180-b90f86320c5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc7cf240f39246578637e6b799607f29', 'request_id': 'req-7d96595d-e001-4d13-a180-b90f86320c5a'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:39.830395 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-7d96595d-e001-4d13-a180-b90f86320c5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc7cf240f39246578637e6b799607f29', 'request_id': 'req-7d96595d-e001-4d13-a180-b90f86320c5a'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:39.885850 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-7d96595d-e001-4d13-a180-b90f86320c5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc7cf240f39246578637e6b799607f29', 'request_id': 'req-7d96595d-e001-4d13-a180-b90f86320c5a'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:39.896715 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-7d96595d-e001-4d13-a180-b90f86320c5a admin admin] The token cache is being invalidate because project fc7cf240f39246578637e6b799607f29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:51:39.897009 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-7d96595d-e001-4d13-a180-b90f86320c5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:39.897656 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-7d96595d-e001-4d13-a180-b90f86320c5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:39.899131 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1458/2910] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:51:39 2022] DELETE /identity/v3/projects/fc7cf240f39246578637e6b799607f29 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:51:39.904013 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ad20cc-a23b-49f1-bd4e-05a89bce0996 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:39.904304 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ad20cc-a23b-49f1-bd4e-05a89bce0996 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:39.904581 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ad20cc-a23b-49f1-bd4e-05a89bce0996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:39.927436 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-86ad20cc-a23b-49f1-bd4e-05a89bce0996 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:51:39.928648 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-86ad20cc-a23b-49f1-bd4e-05a89bce0996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:51:39.988556 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1453/2911] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:51:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:51:39.996528 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c27cbac-ac65-46e1-a6e0-a963224d3a1c None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:39.996819 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c27cbac-ac65-46e1-a6e0-a963224d3a1c None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:39.997100 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c27cbac-ac65-46e1-a6e0-a963224d3a1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:40.017448 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-6c27cbac-ac65-46e1-a6e0-a963224d3a1c None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:51:40.018700 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-6c27cbac-ac65-46e1-a6e0-a963224d3a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:51:40.026604 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1459/2912] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:51:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:51:40.031389 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ad20cc-a23b-49f1-bd4e-05a89bce0996 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:51:40.040899 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ad20cc-a23b-49f1-bd4e-05a89bce0996 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:51:40.045592 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d774140-9e05-4f13-bafd-a38eb1632dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:51:40.046207 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d774140-9e05-4f13-bafd-a38eb1632dfe admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:40.046489 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d774140-9e05-4f13-bafd-a38eb1632dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:40.046771 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d774140-9e05-4f13-bafd-a38eb1632dfe admin admin] PATH_INFO: `/v3/domains` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:40.047426 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d774140-9e05-4f13-bafd-a38eb1632dfe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 07:51:40.047737 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d774140-9e05-4f13-bafd-a38eb1632dfe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:51:40.049327 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d774140-9e05-4f13-bafd-a38eb1632dfe admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:51:40.061032 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1454/2913] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 15 07:51:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 07:51:40.065697 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c27cbac-ac65-46e1-a6e0-a963224d3a1c None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:51:40.074935 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c27cbac-ac65-46e1-a6e0-a963224d3a1c None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:51:40.080212 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee3b975-223f-4255-8772-7e2e727e9709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:51:40.080808 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee3b975-223f-4255-8772-7e2e727e9709 admin admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:40.081093 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee3b975-223f-4255-8772-7e2e727e9709 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:40.081371 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee3b975-223f-4255-8772-7e2e727e9709 admin admin] PATH_INFO: `/v3/projects` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:40.082048 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee3b975-223f-4255-8772-7e2e727e9709 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:51:40.084198 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee3b975-223f-4255-8772-7e2e727e9709 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:51:40.150155 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1460/2914] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 15 07:51:40 2022] POST /identity/v3/projects => generated 399 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 07:51:40.154391 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d774140-9e05-4f13-bafd-a38eb1632dfe admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:51:40.167578 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d774140-9e05-4f13-bafd-a38eb1632dfe admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:51:40.198980 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:51:40.232000 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7151cfd4-f1ed-4587-b85c-fa2b76bfd49e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:51:40.232617 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7151cfd4-f1ed-4587-b85c-fa2b76bfd49e admin admin] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:40.232910 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7151cfd4-f1ed-4587-b85c-fa2b76bfd49e admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:40.233460 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7151cfd4-f1ed-4587-b85c-fa2b76bfd49e admin admin] PATH_INFO: `/v3/users` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:40.234274 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7151cfd4-f1ed-4587-b85c-fa2b76bfd49e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:51:40.236881 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7151cfd4-f1ed-4587-b85c-fa2b76bfd49e admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:51:40.327770 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1455/2915] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 15 07:51:40 2022] POST /identity/v3/users => generated 300 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 07:51:40.331743 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee3b975-223f-4255-8772-7e2e727e9709 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:51:40.344818 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee3b975-223f-4255-8772-7e2e727e9709 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:51:40.349807 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59492be1-f9cb-4365-a659-656af2db8add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:51:40.350453 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59492be1-f9cb-4365-a659-656af2db8add admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:40.350731 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59492be1-f9cb-4365-a659-656af2db8add admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:40.351097 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59492be1-f9cb-4365-a659-656af2db8add admin admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:40.351743 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59492be1-f9cb-4365-a659-656af2db8add admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:51:40.353739 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59492be1-f9cb-4365-a659-656af2db8add admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:51:40.367866 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1461/2916] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:51:40 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 07:51:40.372453 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7151cfd4-f1ed-4587-b85c-fa2b76bfd49e admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:51:40.385607 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7151cfd4-f1ed-4587-b85c-fa2b76bfd49e admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:51:40.393062 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1e1121-bb36-41b6-ba3e-45b5980a236d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:51:40.393715 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1e1121-bb36-41b6-ba3e-45b5980a236d admin admin] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:40.393997 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1e1121-bb36-41b6-ba3e-45b5980a236d admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:40.394272 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1e1121-bb36-41b6-ba3e-45b5980a236d admin admin] PATH_INFO: `/v3/roles` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:40.395289 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1e1121-bb36-41b6-ba3e-45b5980a236d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:51:40.398533 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1e1121-bb36-41b6-ba3e-45b5980a236d admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:51:40.410595 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1456/2917] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 15 07:51:40 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 07:51:40.415016 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59492be1-f9cb-4365-a659-656af2db8add admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:51:40.425330 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59492be1-f9cb-4365-a659-656af2db8add admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:51:40.433011 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde61056-3d32-437b-90bb-bdc61b64c097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:51:40.433702 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde61056-3d32-437b-90bb-bdc61b64c097 admin admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:40.434076 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde61056-3d32-437b-90bb-bdc61b64c097 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:40.434352 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde61056-3d32-437b-90bb-bdc61b64c097 admin admin] PATH_INFO: `/v3/projects/32813e0b7b9a492fbaf022cf740ae068/users/0c0b1683e49e49f5b66fbfe65158f6f3/roles/053e12af84e943e699460a9d76c2072b` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:40.438393 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:51:40.450888 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde61056-3d32-437b-90bb-bdc61b64c097 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32813e0b7b9a492fbaf022cf740ae068, user_id=0c0b1683e49e49f5b66fbfe65158f6f3, role_id=053e12af84e943e699460a9d76c2072b)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:51:40.454495 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde61056-3d32-437b-90bb-bdc61b64c097 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:51:40.651690 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1462/2918] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 15 07:51:40 2022] PUT /identity/v3/projects/32813e0b7b9a492fbaf022cf740ae068/users/0c0b1683e49e49f5b66fbfe65158f6f3/roles/053e12af84e943e699460a9d76c2072b => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:51:40.652446 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c648fdd-b1a8-464d-be98-9ed64c2c94a6 None None] REQUEST_METHOD: `POST` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:40.652446 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c648fdd-b1a8-464d-be98-9ed64c2c94a6 None None] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:40.652446 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c648fdd-b1a8-464d-be98-9ed64c2c94a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:40.663292 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.auth.core [None req-7c648fdd-b1a8-464d-be98-9ed64c2c94a6 None None] MFA Rules not processed for user `0c0b1683e49e49f5b66fbfe65158f6f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:51:40.665230 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-7c648fdd-b1a8-464d-be98-9ed64c2c94a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:51:40.735717 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1457/2919] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:51:40 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 07:51:40.748004 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde61056-3d32-437b-90bb-bdc61b64c097 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:51:40.751471 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-dde61056-3d32-437b-90bb-bdc61b64c097 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:51:40.818864 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde61056-3d32-437b-90bb-bdc61b64c097 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:51:40.822602 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311bd6cd-91e2-462f-ac10-465a323e482b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecf6642b7b84d3c9756bb41ab3349d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:51:40.823241 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311bd6cd-91e2-462f-ac10-465a323e482b service cinder] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:40.823725 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311bd6cd-91e2-462f-ac10-465a323e482b service cinder] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:40.824232 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311bd6cd-91e2-462f-ac10-465a323e482b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:40.835576 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311bd6cd-91e2-462f-ac10-465a323e482b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:51:40.837184 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311bd6cd-91e2-462f-ac10-465a323e482b service cinder] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:51:40.851400 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1463/2920] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:51:40 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:51:48.925151 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c648fdd-b1a8-464d-be98-9ed64c2c94a6 None None] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:51:48.989747 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c648fdd-b1a8-464d-be98-9ed64c2c94a6 None None] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:51:48.994270 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bedf85d-264f-4ce3-bd58-38f71185df3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:51:48.994900 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bedf85d-264f-4ce3-bd58-38f71185df3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:48.995183 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bedf85d-264f-4ce3-bd58-38f71185df3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:48.995457 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bedf85d-264f-4ce3-bd58-38f71185df3a admin admin] PATH_INFO: `/v3/users/0c0b1683e49e49f5b66fbfe65158f6f3` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:48.996578 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bedf85d-264f-4ce3-bd58-38f71185df3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c0b1683e49e49f5b66fbfe65158f6f3)` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:51:48.999177 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bedf85d-264f-4ce3-bd58-38f71185df3a admin admin] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:51:49.165840 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:51:49.174688 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-7bedf85d-264f-4ce3-bd58-38f71185df3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c0b1683e49e49f5b66fbfe65158f6f3'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:49.175342 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-7bedf85d-264f-4ce3-bd58-38f71185df3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c0b1683e49e49f5b66fbfe65158f6f3'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:49.262605 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-7bedf85d-264f-4ce3-bd58-38f71185df3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c0b1683e49e49f5b66fbfe65158f6f3'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:49.263226 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-7bedf85d-264f-4ce3-bd58-38f71185df3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c0b1683e49e49f5b66fbfe65158f6f3'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:49.271192 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.notifications [None req-7bedf85d-264f-4ce3-bd58-38f71185df3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c0b1683e49e49f5b66fbfe65158f6f3'} {{(pid=81597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:49.288574 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1458/2921] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 15 07:51:48 2022] DELETE /identity/v3/users/0c0b1683e49e49f5b66fbfe65158f6f3 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:51:49.293005 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60103442-9570-4247-ace0-6ec59e172660 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:49.293516 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60103442-9570-4247-ace0-6ec59e172660 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:49.293658 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60103442-9570-4247-ace0-6ec59e172660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:49.313714 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-60103442-9570-4247-ace0-6ec59e172660 None None] MFA Rules not processed for user `06cea60dcdfb4c5ead6de8db2f0fd82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 07:51:49.314902 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-60103442-9570-4247-ace0-6ec59e172660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:51:49.374109 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1464/2922] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 15 07:51:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 07:51:49.383639 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bedf85d-264f-4ce3-bd58-38f71185df3a admin admin] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:51:49.385028 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-7bedf85d-264f-4ce3-bd58-38f71185df3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:51:49.443196 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bedf85d-264f-4ce3-bd58-38f71185df3a admin admin] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:51:49.446777 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39426e04-dd25-4a81-89ab-b3c803b97d6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5315996432f44bffa3460b2744c10f0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:51:49.447587 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39426e04-dd25-4a81-89ab-b3c803b97d6a service neutron] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:49.447710 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39426e04-dd25-4a81-89ab-b3c803b97d6a service neutron] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:49.447910 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39426e04-dd25-4a81-89ab-b3c803b97d6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:49.459738 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39426e04-dd25-4a81-89ab-b3c803b97d6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:51:49.471253 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39426e04-dd25-4a81-89ab-b3c803b97d6a service neutron] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:51:49.471651 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1459/2923] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:51:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:51:49.905699 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60103442-9570-4247-ace0-6ec59e172660 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:51:49.910507 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-60103442-9570-4247-ace0-6ec59e172660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:51:49.922777 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60103442-9570-4247-ace0-6ec59e172660 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:51:49.930386 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be4e7ae-28d4-428b-90d0-1b671d6ebf16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cea60dcdfb4c5ead6de8db2f0fd82f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918580af0a81452f8da95d60256aaea0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:51:49.931455 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be4e7ae-28d4-428b-90d0-1b671d6ebf16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:49.931969 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be4e7ae-28d4-428b-90d0-1b671d6ebf16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:49.932460 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be4e7ae-28d4-428b-90d0-1b671d6ebf16 admin admin] PATH_INFO: `/v3/projects/32813e0b7b9a492fbaf022cf740ae068` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:49.934266 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be4e7ae-28d4-428b-90d0-1b671d6ebf16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32813e0b7b9a492fbaf022cf740ae068)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:51:49.937435 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be4e7ae-28d4-428b-90d0-1b671d6ebf16 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:51:50.057508 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-6be4e7ae-28d4-428b-90d0-1b671d6ebf16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32813e0b7b9a492fbaf022cf740ae068', 'request_id': 'req-6be4e7ae-28d4-428b-90d0-1b671d6ebf16'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:50.063657 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-6be4e7ae-28d4-428b-90d0-1b671d6ebf16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32813e0b7b9a492fbaf022cf740ae068', 'request_id': 'req-6be4e7ae-28d4-428b-90d0-1b671d6ebf16'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:50.142854 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-6be4e7ae-28d4-428b-90d0-1b671d6ebf16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32813e0b7b9a492fbaf022cf740ae068', 'request_id': 'req-6be4e7ae-28d4-428b-90d0-1b671d6ebf16'} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:50.153782 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-6be4e7ae-28d4-428b-90d0-1b671d6ebf16 admin admin] The token cache is being invalidate because project 32813e0b7b9a492fbaf022cf740ae068 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 07:51:50.154080 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-6be4e7ae-28d4-428b-90d0-1b671d6ebf16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:50.155424 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [None req-6be4e7ae-28d4-428b-90d0-1b671d6ebf16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 07:51:50.156892 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1465/2924] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 15 07:51:49 2022] DELETE /identity/v3/projects/32813e0b7b9a492fbaf022cf740ae068 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 07:51:51.056632 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39426e04-dd25-4a81-89ab-b3c803b97d6a service neutron] Authenticating user token {{(pid=81597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:51:51.058072 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-39426e04-dd25-4a81-89ab-b3c803b97d6a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:51:51.129127 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39426e04-dd25-4a81-89ab-b3c803b97d6a service neutron] Validating token access rules against request {{(pid=81597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:51:51.181979 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:51:51.190450 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a0cc5e-e451-420f-ae34-08db10e978b9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd87846b9a134248b28e0521b0a880c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:51:51.191064 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a0cc5e-e451-420f-ae34-08db10e978b9 service placement] REQUEST_METHOD: `GET` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:51:51.192997 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a0cc5e-e451-420f-ae34-08db10e978b9 service placement] SCRIPT_NAME: `/identity` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:51:51.192997 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a0cc5e-e451-420f-ae34-08db10e978b9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:51:51.193567 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.fernet_utils [None req-47a0cc5e-e451-420f-ae34-08db10e978b9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:51:51.205377 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a0cc5e-e451-420f-ae34-08db10e978b9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:51:51.206916 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a0cc5e-e451-420f-ae34-08db10e978b9 service placement] RBAC: Authorization granted {{(pid=81597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:51:51.217345 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: [pid: 81597|app: 0|req: 1460/2925] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:51:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:52:00.150649 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:52:01.274556 dv-cinder-824149-6-iscsi devstack@keystone.service[81597]: DEBUG dbcounter [-] [81597] Writing DB stats keystone:SELECT=5 {{(pid=81597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 07:56:59.107590 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be4e7ae-28d4-428b-90d0-1b671d6ebf16 admin admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 07:56:59.108963 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-6be4e7ae-28d4-428b-90d0-1b671d6ebf16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 07:56:59.191962 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be4e7ae-28d4-428b-90d0-1b671d6ebf16 admin admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 07:56:59.198333 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d04b0c-6342-4568-9e04-3e66300153df service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd87846b9a134248b28e0521b0a880c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e9987720930473da840579654311bc2', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 07:56:59.198880 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d04b0c-6342-4568-9e04-3e66300153df service placement] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 07:56:59.199163 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d04b0c-6342-4568-9e04-3e66300153df service placement] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 07:56:59.199444 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d04b0c-6342-4568-9e04-3e66300153df service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 07:56:59.208132 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d04b0c-6342-4568-9e04-3e66300153df service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 07:56:59.210033 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d04b0c-6342-4568-9e04-3e66300153df service placement] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 07:56:59.220998 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1466/2926] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 15 07:56:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 07:57:09.216929 dv-cinder-824149-6-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=37 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}